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Abstract 


In  June  of  1995,  senior  Air  Force  leadership  led  by  Secretary  Widnall  and  General 
Ronald  Fogelman,  Chief  of  Staff  met  to  establish  policy  for  the  future  of  Air  Force 
modeling  and  simulation  (M&S).  The  result  of  this  conference  was  a  the  publication  of  A 
New  Vector  as  a  balanced  strategy  and  a  roadmap  for  Air  Force  M&S.  The  M&S 
roadmap  in  A  New  Vector  is  best  compared  to  something  with  which  the  Air  Force  is  very 
familiar — how  to  assess  and  invest  in  the  capability  of  aircraft  systems.  Because  aircraft 
systems  assessment  is  the  basic  mission  of  flight-test,  this  thesis  addresses  the  question: 
As  design  and  engineering  models  increase  in  scope  and  complexity  from  the  physical 
component  level  to  complex  systems,  what  are  the  capabilities  and  limitations  of  M&S 
for  flight  test? 

In  order  to  answer  the  research  question  three  criteria  were  used  to  evaluate  the 
results  of  each  case  study.  First,  the  costs  and  benefits  of  M&S  were  compared  with  the 
tradeoffs  required  for  not  doing  M&S.  Second,  the  engineering  validity  of  the  M&S 
efforts  was  evaluated.  Engineering  validity  is  the  determination  as  to  whether  the 
modeling  data  were  sufficiently  accurate  to  solve  the  problem  at  hand.  Third,  the 
comprehensiveness  of  each  M&S  effort  was  evaluated.  For  the  purposes  of  this  study, 
comprehensiveness  is  whether  an  M&S  effort  had  sufficient  scope  to  solve  the  entire 
problems.  An  M&S  effort  that  completely  and  accurately  predicted  the  experimental 
outcome  of  flight-test  was  comprehensive. 

The  process  used  for  answering  these  questions  was  a  systematic  examination  of 
three  case  studies.  The  first  case  covered  M&S  at  the  physical  component  level,  and 
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looked  at  how  computational  fluid  dynamics  was  used  to  solve  a  stores  separation 
problem  on  the  B-1B.  The  second  case  was  an  examination  of  the  VISTA  in-flight 
simulator  as  a  tool  for  modeling  closed-loop  control  systems.  The  third  case  is  examines 
the  role  of  M&S  in  designing  and  flight-testing  the  Boeing  777. 

All  three  cases  discussed  in  this  research  have  made  significant  progress  towards 
meeting  the  challenges  set  out  by  senior  Air  Force  leadership  in  A  New  Vector.  Although 
M&S  is  not  free,  it  is  usually  more  cost-effective  than  testing  alone.  The  engineering 
validity  of  M&S  was  excellent  in  all  three  cases.  The  accuracy  of  the  models  was  equal  to 
or  better  than  traditional  engineering  methods.  However,  as  the  modeling  efforts  became 
more  complex,  it  was  not  possible  to  build  comprehensive  models  of  Boeing  777  with  the 
current  state  of  the  art.  As  M&S  becomes  more  integrated  into  the  aircraft  design  process, 
the  Air  Force  must  better  understand  the  capabilities  and  limitations  of  M&S  before 
flight-testing. 
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Chapter  1 


Envisioning  the  Future  of  Flight-Test  with  Modeling  and 

Simulation 

Never  again  will  the  United  States  ride  the  coattails  of  other  countries  in 
the  progress  and  development  of  the  aeronautical  art... 

President  Harry  Truman 
June  25,  1951 

As  Commanding  General  of  the  Army  Air  Forces  during  World  War  II,  General  Hap 
Arnold  not  only  saw  the  need  for  a  separate  Air  Force,  but  he  was  also  concerned  about 
the  role  of  technology  and  scientific  research  in  developing  airpower.  After  achieving  a 
separate  Air  Force  in  1947,  General  Arnold  wrote:  “I  have  yet  another  job.  That  is  to 
project  myself  into  the  future... and  determine  the  steps  the  US  should  take  to  have  the 
best  Air  Force  in  the  world  20  years  hence...”1  General  Arnold’s  intention  was  clear.  The 
United  States  would  maintain  the  most  capable  and  technologically  advanced  Air  Force 
in  the  world.  General  Arnold’s  vision  continues  to  have  a  profound  influence  on  the  Air 
Force  today. 

Almost  fifty  years  later,  senior  Air  Force  leadership  continues  General  Arnold’s 
tradition  by  projecting  into  the  future  and  determining  the  steps  necessary  to  be  the  best 
Air  Force  in  the  world.  Fortunately,  the  US  Air  Force  currently  holds  the  technological 
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lead  in  almost  every  aspect  of  the  aeronautical  arts.  However,  rapid  advances  in 
information  technology  threaten  this  lead  by  reducing  the  lead-time  required  to  go  from 
initial  concept  to  fielded  military  capability.  In  order  for  the  Air  Force  to  sustain  its 
technological  lead  in  the  aeronautical  arts,  it  must  now  move  to  the  forefront  of  the 
information  technology  revolution  as  well. 

Information  technology,  especially  Modeling  &  Simulation  (M&S)  provides 
powerful  engineering  and  design  tools  to  shape  the  future  of  airpower.  Present  trends 
indicate  that  M&S  technology  will  become  an  increasingly  powerful  aspect  of 
aeronautical  engineering,  design,  and  flight-te sting.  Currently  computer  speeds  are 
doubling  every  18  to  24  months  while  the  costs  of  computing  power  are  being  halved 
during  the  same  period.  This  trend  is  likely  to  continue  for  the  near  future.  Therefore, 
senior  Air  Force  leadership  decided  to  take  the  steps  necessary  to  move  the  US  Air  Force 
to  the  forefront  of  M&S  technology. 

In  early  June  1995,  the  senior  Air  Force  leadership,  led  by  the  Secretary  of  the  Air 
Force  Sheila  Widnall  and  the  Chief  of  Staff,  General  Ronald  Fogleman,  gathered  to  set  A 
New  Vector  for  the  future  of  Air  Force  M&S.  The  result  of  this  meeting  was  a 
“roadmap”  for  the  future  of  Air  Force  M&S  capabilities  in  the  21st  century.  Senior 
leadership  has  endorsed  the  Air  Force  M&S  road  map — as  a  balanced  strategy  for  a  clear 
vision  of  the  future.2  With  these  actions,  the  Air  Force  took  a  critical  step  toward  having 
the  most  technologically  advanced  Air  Force  in  the  next  century. 

Notes 

1  Vision — The  History  and  Mission  of  Arnold  Engineering  Development  Center.  (Arnold 
AFB,TN.:  Public  Affairs  Office,  Arnold  Engineering  Development  Center.  February 
1994),  2. 

“  Air  Force  Four  Star  Commission  on  Modeling  and  Simulation.  A  New  Vector:  Air 
Force  Modeling  and  Simulation  (Washington,  DC:  Government  Printing  Office,  1995),  1. 
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Since  the  publication  of  A  New  Vector ,  the  aeronautical  arts  have  made  significant 
advances  as  the  result  of  M&S  technologies,  and  the  Air  Force  has  continued  to  move  a 
little  closer  to  the  vision  outlined  by  senior  leadership.  The  vision  of  A  New  Vector  is 
based  on  a  four-tiered  roadmap  that  comprises  the  elements  of  vision,  quality,  people,  and 
infrastructure.  The  framework  for  this  roadmap  is  shown  in  Figure  1  below.  While  this 
roadmap  draws  attention  to  critical  elements,  it  is  only  notional  and  not  intended  to 
provide  detailed  instructions  or  specific  examples. 

The  New  Vector  roadmap  provides  a  broad  view  of  current  M&S  initiatives.  It 
provides  the  overview  and  lays  out  the  Air  Force’s  institutional  priorities.  However,  A 
New  Vector  does  not  provide  specific  guidance  about  how  and  where  the  Air  Force  must 
begin  grappling  with  M&S  technologies.  Despite  the  broad  policy  guidance  in  A  New 
Vector ,  most  Air  Force  M&S  efforts  remain  specialized  engineering  tools.  Therefore,  the 
Air  Force  cannot  fully  exploit  the  remarkable  synergies  of  M&S  technologies  for 
complex  systems. 


Standard* 

Integration 


Manpowei 


infrastructure 


AOS  Leverago 


Figure  1:  The  Modeling  and  Simulation  Roadmap 
from  A  New  Vector 

This  thesis  answers  the  question  as  to  how  and  where  the  Air  Force  must  begin 
grappling  with  the  specifics  of  integrating  M&S  technology  into  the  process  of  designing 
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and  flight-testing  aircraft.  M&S  is  already  a  powerful  engineering  and  design  tool.  Flight- 
test  is  the  final  evaluation  of  the  entire  aircraft  design  and  production  process,  and  aircraft 
systems  assessment  is  the  basic  mission  of  flight-test.  As  M&S  becomes  more  integrated 
into  the  aircraft  design  process,  the  Air  Force  must  understand  the  capabilities  and 
limitations  of  M&S  before  flight-testing.  Therefore,  this  thesis  addresses  the  question:  As 
design  and  engineering  models  increase  in  scope  and  complexity  from  the  physical 
component  level  to  complex  systems,  what  are  the  capabilities  and  limitations  of  M&S 
for  flight-test? 

The  process  used  for  answering  these  questions  is  a  systematic  examination  of  three 
case  studies.  The  case  studies  were  chosen  because  together  they  cover  a  wide  spectrum 
of  M&S  capabilities  from  physical  component  models  to  models  of  highly  complex 
systems.  Each  case  illuminates  some  of  the  extraordinary  capabilities  as  well  as  the 
inherent  limitations  of  M&S.  The  cases  also  show  that  M&S  is  now  a  critical  aspect  of 
major  weapons  system  development,  and  rapid  technological  advances  in  M&S  are 
making  a  profound  impact  on  the  way  weapons  systems  are  being  developed  and  tested.3 

In  order  to  answer  the  research  question  three  criteria  will  be  used  to  evaluate  the 
results  of  each  case  study.  First,  the  costs  and  benefits  of  M&S  will  be  compared  to  the 
tradeoffs  required  for  not  doing  M&S.  Second,  the  engineering  validity  of  the  M&S 
efforts  will  be  evaluated.  Engineering  validity  is  the  determination  as  to  whether  the 
modeling  data  were  sufficiently  accurate  to  solve  the  problem  at  hand.  Third,  the 
comprehensiveness  of  each  M&S  effort  will  be  evaluated.  For  the  purposes  of  this  study, 
comprehensiveness  is  whether  an  M&S  effort  has  sufficient  scope  to  solve  the  entire 
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problems.  An  M&S  effort  that  completely  and  accurately  predicts  the  experimental 
outcome  of  flight-test  is  comprehensive.  On  the  other  hand,  an  M&S  effort  that  makes 
errors  of  omission  or  does  not  account  for  significant  experimental  results  is  not  a 
comprehensive  model. 

The  first  case  study  demonstrates  the  capabilities  and  limitations  of  M&S  at  the 
physical  component  level.  In  this  case,  computational  fluid  dynamics  (CFD)  was  used  to 
solve  a  B-1B  flare  strike  problem  by  modeling  the  aircraft  and  the  flare  as  physical 
components  in  the  flight  environment.  This  problem  restricted  flare  operations  and 
significantly  reduced  the  combat  capability  of  the  B- IB  weapons  system.4  Yet,  M&S  and 
played  an  essential  role  in  augmenting  flight-test  efforts  and  resolving  the  problem  — 
restoring  the  fleet  to  full  operations. 

The  second  case  looks  at  the  Variable  Stability  In-flight  Test  Aircraft  (VISTA).  The 
VISTA  is  a  highly  modified  F-16  fighter  aircraft  that  can  actually  simulate  the  handling 
qualities  of  another  aircraft  in-flight.  The  VISTA  aircraft  uses  M&S  technology  to  alter 
its  flying  qualities  for  the  pilot.  The  software  and  methodology  is  very  similar  to  a  ground 
simulator,  except  that  the  variable  stability  system  is  sophisticated  enough  to  change  the 
actual  response  of  the  aircraft  to  various  control  inputs.  In  contrast  to  the  B-1B  flare 
strike  case  study  that  modeled  physical  components,  the  VISTA  modeled  a  complete 
aircraft  with  man-in-the-loop  interactions,  which  significantly  increased  the  complexity 
of  the  M&S  effort. 


Notes 

Department  of  Defense  (DOD)  Directive  5000.59.  Department  of  Defense  (DOD) 
Modeling  and  Simulation  ManagementA  January  1994.2. 

4  By  order  of  the  ACC/DO  March  1995. 
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The  VISTA  was  used  to  help  design  and  test  the  F-22  flight  control  system.  The  fly¬ 
by-wire  flight  control  system  on  the  F-22  is  the  most  sophisticated  flight  control  system 
ever  engineered.  The  F-22’s  flight  controls  have  incorporated  state  of  the  art  digital 
control  theory  to  improve  its  flying  qualities  as  well  as  make  the  fly-by-wire  system 
robust  and  failure  tolerant.  Fly-by-wire  systems  are  themselves  model-following  control 
systems.  Therefore,  the  desired  response  of  the  aircraft  is  written  into  the  computer  code 
as  a  model.  The  VISTA  allows  that  code  to  be  flight-tested  carefully  in  order  to  find 
software  “bugs,”  improve  flying  qualities,  and  to  increase  the  control  systems  failure 
tolerance.  Because  the  VISTA  has  a  redundant  flight  control  system,  flight-testers  can 
test  fly-by-wire  control  systems  without  endangering  a  pilot  or  valuable  prototype 
aircraft. 

The  third  case  study  examines  how  the  Boeing  Company  used  M&S  technology  to 
make  radical  changes  in  how  they  design,  produce,  and  flight-test  airplanes.  The  Boeing 
777  was  the  first  commercial  jet  completely  designed  and  prototyped  by  using  M&S 
technology.  The  changes  wrought  by  M&S  technologies  were  very  dramatic  and 
encompassed  many  areas,  including  technical,  organizational  and  administrative  changes. 
Although  the  technical  innovations  were  numerous,  the  Boeing  777  project  was  unique 
because  of  the  central  role  of  M&S  technology  in  the  design,  construction,  and  testing  of 
the  airplane. 

Boeing  used  M&S  technology  to  completely  design  and  prototype  an  aircraft  in 
cyberspace.  Instead  of  engineering  drawings,  the  Boeing  777  design  team  used  three- 
dimensional  solid  modeling  technology.  The  software  used  for  this  effort  was  CATIA 
(Computer-Aided,  Three-dimensional  Interactive  Application)  developed  by  Dassault 


6 


Systems  of  France.  The  Boeing  111  division  used  more  than  2,200  CATIA  workstations 
networked  to  a  mainframe  computing  cluster.  The  project  was  the  largest  ever  computer- 
aided  design  (CAD)  project.  Concurrent  engineering  was  also  an  integral  part  of  Boeing’s 
M&S  effort.  Each  “Design/Build”  team  interacted  extensively  with  the  others  through 
their  CATIA  workstations  and  coordinated  design  changes  by  sharing  three-dimensional 
models  that  were  used  by  all  the  other  teams.  These  teams  included  representatives  from 
design,  engineering,  manufacturing,  customers,  and  suppliers. 

In  the  past,  Boeing  designers  had  always  built  a  full-scale  non-flying  mockup  of  the 
complete  aircraft  to  check  form,  fit,  and  interference.  Since  various  aircraft  systems  were 
designed  independently,  it  was  necessary  to  make  painstakingly  precise  physical 
mockups.  This  practice  was  very  expensive  and  the  various  required  mockups  of  the 
Boeing  111  were  estimated  to  have  cost  more  than  $22.5  million.  The  mockup  was  also 
used  to  check  accessibility  of  all  the  parts  for  maintenance  work.  However,  with  a  three- 
dimensional  database  that  everyone  on  the  design  team  could  use  simultaneously, 
prototype  phase  could  be  bypassed  in  favor  of  going  directly  into  production. 

Because  the  production  aircraft  was  built  to  conform  to  computer  models,  the  B-777 
flight-test  program  was  essentially  a  model  verification  program.  This  case  study  is  an 
extremely  important  example  of  how  M&S  technology  can  cause  enormous  impacts  from 
the  initial  design  through  flight-testing.  There  are  many  lessons  the  Air  Force  can  learn 
from  the  Boeing  experience  in  order  to  implement  the  policy  guidance  from  A  New 
Vector. 

Each  case  in  this  research  is  an  important  example  of  M&S  when  considered  alone. 
However,  the  implications  for  the  future  of  aircraft  design  and  flight-test  are  enormous 
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when  the  capabilities  illustrated  by  each  case  are  considered  together.  M&S  is  now  an 
essential  part  of  the  total  aircraft  system  design,  engineering,  and  test  effort.  As  part  of 
the  roadmap  from  A  New  Vector  for  the  21st  century,  the  Air  Force  will  be  integrating 
M&S  into  the  engineering,  design,  and  flight-test  in  order  to  develop  21st  century 
airpower. 
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Chapter  2 


Case  Study  One-Physical  Component  Modeling 
of  the  B-1B  Flare  System 


Testing  is  the  conscience  of  acquisitions... 

Honorable  Phillip  E.  Coyle  III 
Undersecretary  of  Defense 

Introduction 

This  case  study  covers  a  major  success  of  M&S  at  the  physical  component  level.  The 
case  involved  a  problem  concerning  flare  strikes  that  the  B-1B  operational  fleet  was 
suffering.  The  damage  caused  by  the  flares  striking  the  aircraft  restricted  flare  operations 
and  caused  a  flight  safety  problem.  Because  the  flare  strikes  seemed  to  be  random 
occurrences,  classical  flight-test  techniques  could  not  solve  the  problem  alone.  M&S 
techniques  were  essential  to  finding  the  underlying  causes  of  the  flare  strikes. 
Specifically,  computational  fluid  dynamics  (CFD) — a  sophisticated  method  of  modeling 
the  forces  acting  on  a  body  in  a  flowfield —  was  used  to  resolve  the  problem  and  restore 
the  B-1B  fleet  to  full  operations. 

Historical  Background  of  the  B-1B  Flare  Problem 

The  B-1A  was  originally  designed  as  a  high  altitude,  supersonic,  strategic  nuclear 
bomber.  President  Jimmy  Carter  canceled  the  B-1A  program  in  June  1977,  but  President 
Ronald  Reagan  later  revived  the  program  as  the  B-1B  in  October  of  1981.  However,  the 
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mission  changed  due  to  emerging  threats  in  the  early  1980’s  and  the  B-1B  took  on  the 
role  of  a  low-level  strategic  nuclear  bomber  with  high  subsonic  ingress  and  egress 
capabilities,  as  well  as  a  supersonic  dash  capability.5  The  B-1B  defensive  systems  were 
redesigned  for  its  new  mission,  with  the  primary  aircraft  threat  now  evolving  from  a  radar 
guided  missile  to  a  look  down  infrared  (IR)  missile  attack.  Therefore,  one  of  the 
important  defensive  systems  retained  from  the  B-1A  program  was  the  MJU-23/B  IR 
flare.  The  MJU-23B  flare  was  designed  to  defeat  both  the  surface-to-air  and  air-to-air 
heat-seeking  missiles  employed  by  potential  adversaries  (see  Figure  2:  MJU-23/B  Flare 
Used  on  the  B-1B). 

Subsequent  to  this  design  point,  the  mission  of  the  aircraft  was  again  changed  to  a 
conventional  bombing  role.  As  a  conventional  bomber,  the  B-1B  must  operate  without  a 
nuclear  “lay-down”  of  enemy  air  defenses  from  ICBM  attacks,  and  the  aircraft  must  be 
able  to  survive  in  a  high  threat  environment.  Because  of  the  change  to  a  conventional 
mission,  B-1B  tactical  doctrines  were  changed  to  reflect  the  new  threat  environment  and 
new  tactics  were  developed.6  Consequently,  Air  Combat  Command  (ACC)  substantially 
improved  its  training  scenarios  to  reflect  the  B-lB’s  evasive  combat  maneuvers  and  the 
anticipated  flare  use.  These  factors  all  contributed  to  the  increased  use  of  flares,  as  well 
as  the  resulting  aircraft  strikes. 

The  B-1B  operational  fleet  began  suffering  strikes  from  the  MJU-23/B  defensive 
flares  with  an  alarming  frequency  from  1994  through  early  1995.  These  flare  strikes  took 
two  different  forms:  “major”  strikes  and  “minor”  strikes.  Major  strikes  were  defined  as 

Note 

5  Major  Richard  A.  Stuckey  and  Major  Craig  T.  Edkins,  “B-1B  Flight-Test  Update,” 
Proceedings  of  the  39th  Symposium  of  the  Society  of  Experimental  Test  Pilots,  September 
1995. 
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any  flare  strikes  that  produced  large  holes,  burns,  or  damage  to  composite  surfaces. 
Minor  strikes  were  defined  as  any  strike  that  produced  only  scuffs,  paint  scrapes,  or  small 
punctures  in  the  skin.  Generally,  major  strikes  only  occurred  on  the  horizontal  and 
vertical  tail  surfaces,  while  minor  strikes  were  found  all  along  the  upper  surface  of  the 
fuselage  and  over  the  entire  empennage  area.  As  a  direct  result  of  these  strikes — and  their 
increasing  frequency — the  ACC  Deputy  for  Operations  (ACC/DO)  ordered  the 
termination  of  all  training  activities  with  live  flares  in  March  1995. 7  The  training 
moratorium  was  to  stay  in  effect  until  a  solution  to  the  problem  could  be  discovered  and 
implemented. 

The  B-1B  System  Program  Office  (SPO)  at  Wright-Patterson  AFB,  OH  and  the  B- 
1B  System  Sustainment  Office  (SSO)  at  the  Oklahoma  Air  Logistics  Center  (OC-ALC)  at 
Tinker  AFB,  OK  were  notified  and  began  the  effort  to  solve  this  problem.  Coincidentally, 
OC-ALC  had  already  begun  work  on  a  MJU-23/B  flare  modification  program  with  the 
flare  item  managers  at  the  Ogden  Air  Logistics  Center  (OO-ALC),  Hill  AFB,  UT  and 
with  Tracor  Corporation  in  San  Ramon,  CA,  the  flare  manufacturer.  The  flare 
modification  program  was  primarily  focused  on  production  and  cost-reduction  issues. 
However,  the  increasing  flare  strike  threat  to  the  fleet  aircraft  provided  an  opportunity  for 
a  synergized  approach. 

OC-ALC  arranged  for  flight-test  activity  with  the  419th  Flight  Test  Squadron 
(FLTS)  at  the  Air  Force  Flight  Test  Center  (AFFTC),  Edwards  AFB,  CA,  which  served 
as  the  B-1B  Combined  Test  Force  (CTF).  Both  the  OC-ALC  and  the  CTF’s  first  major 
objective  was  to  determine  and  model  the  trajectory  of  the  flares,  thereby  identifying 

Notes 

6  T.O.  IB- IB- 1-2,  Offensive/Defensive  Systems  Officers  Flight  Manual,  15  Aug  94. 
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potentially  risky  flight  conditions.  In  order  to  accomplish  this  goal,  the  CTF  also 
requested  M&S  support  from  the  Arnold  Engineering  Development  Center  (AEDC)  at 
Arnold  AFB,  TN.  AEDC  had  extensive  experience  in  M&S  and  had  recently  completed 
a  stores  separation  analysis  for  the  F-22  program  using  CFD  modeling  methods  that  were 
applicable  to  the  B-1B  flare  strike  problem. 

The  B-1B  CTF  also  requested  support  from  the  Naval  Air  Warfare  Center,  Weapons 
Division  (NAWCWPNS)  because  of  their  extensive  flight-test  experience  with  flare 
trajectory  mapping  and  modeling.  These  six  organizations  (OC-ALC,  OO-ALC,  CTF, 
AEDC,  NAWCWPNS,  and  Tracor)  quickly  integrated  themselves  into  a  cohesive  team. 
The  team  combined  flight-test  techniques,  practical  engineering  experience,  and  state  of 
the  art  CFD  modeling  methods  to  determine  the  cause  of  the  flare  strikes  on  the  B-1B 
fleet  aircraft. 

The  use  of  the  MJU-23/B  flare  was  unique  to  the  B-1B  program  — no  other  weapons 
system  in  the  world  employed  anything  like  it.  Each  cylindrical  flare  pellet  weighed  over 
3.25  pounds.  The  flares  were  deployed  from  eight  dispensers  with  twelve  flares  each, 
which  were  located  on  the  top  of  the  aircraft  just  aft  of  the  crew  compartment  (see  Figure 
3:  Flare  Canister  Focations  on  the  B-1B).8  The  flares  were  loaded  into  individual 
canisters  that  were  equipped  with  an  ejection  cartridge,  as  well  as  seals  and  covers  for 
environmental  protection  in  flight  and  on  the  ground.  These  canisters  were  then  loaded 
into  the  dispensers  in  the  maintenance  backshops,  and  the  dispensers  were  loaded  onto 
the  aircraft  at  the  flight  line. 


Notes 

7  By  Order  of  ACC/DO,  1  March  1995 

8  T.O.  IB- IB- 1-2,  Offensive/Defensive  Systems  Officers  Flight  Manual,  15  Aug  94. 
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Figure  2:  MJU-23/B  Flare  Used  on  the  B-1B 


The  original  MJU-23/B  flare  design  specification,  created  during  the  original  B-1A 
program,  dictated  that  the  flare  ejection  velocity  remain  between  80-  feet  per  second  (fps) 
and  120-fps.9  The  flare  ejection  velocities  were  measured  for  specification  compliance 
during  sled  track  tests  at  Hollomon  AFB  in  the  early  1980’s.  The  flares  were  also  flight- 
tested  successfully  at  Edwards  in  straight  and  level  flight  during  the  same  period.  None  of 
these  tests  indicated  any  difficulties.10  However,  the  design  specifications  and  flight-test 
methods  were  designed  for  the  non-maneuvering  B-1A  at  high  altitude  instead  of  the 
low-altitude,  high  speed,  and  aggressive  maneuvers  the  B-1B  used  in  thel990’s.  New 


Note 

9  419  Flight  Test  Squadron,  “MJU-23B  Flare  Trajectory  Analysis  Test  Plan,”  Edwards 
AFB,  CA^  March  1995.  23. 

10  419  Flight  Test  Squadron,  “Test  Director’s  Flight  Test  Fogs”  Edwards  AFB,  CA. 
November  1983.  microfiche. 
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flight-test  techniques  would  be  required  to  model  flare  trajectories  relative  to  a 
maneuvering  aircraft. 

Experimental  Techniques 

The  original  request  for  flare  testing  from  OC-ALC  to  the  CTF  was  only  to  evaluate 
a  seemingly  insignificant  flare  modification.  Approval  for  such  a  test  was  obtained  at 
AFFTC.  However,  a  secondary  objective  soon  followed  to  model  flare  trajectories,  with  a 
particular  request  to  observe  and  record  any  damage  to  the  aircraft  during  the  flight-test 
program.  The  CTF  then  investigated  the  background  of  this  secondary  request.  This 
investigation  revealed  the  history  of  the  flare  strikes  mentioned  previously.  The  flight-test 
plan  was  reviewed  and  the  CTF  determined  that  this  new  objective  took  the  project  well 
beyond  both  the  technical  and  safety  scope  that  had  been  approved  by  AFFTC. 


Figure  3:  Flare  Canister  Locations  on  B-1B 

The  flight-test  plan  was  rewritten  to  include  the  flare  damage  investigation,  but  the 
CTF  quickly  realized  the  need  for  analysis  before  they  could  safely  determine  the  cause 
of  the  problem.  The  CTF  requested  that  OC-ALC  enlist  analytical  support,  realizing  the 
need  to  gain  a  technical  understanding  of  the  problem  facing  them.  OC-ALC  responded 
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by  contracting  with  AEDC  and  its  support  contractor,  the  Sverdrup  Corporation,  to 
perform  M&S  of  the  flare  trajectories  using  CFD. 

The  CTF  tasked  AEDC  to  evaluate  two  specific  aspects  of  the  flare  strikes  on  the  B- 
1B  empennage.  First,  AEDC  was  tasked  to  look  at  the  flowfield  around  the  aircraft  and 
to  identify  any  anomalies  that  may  have  been  depressing  the  flare  trajectory,  such  as  wing 
tip  vortices.  Second,  AEDC  was  tasked  to  look  at  maneuvering  flight  in  order  to 
determine  if  any  unusual  characteristics  appeared  in  the  “aggravated”  flowfield.  In  order 
to  accomplish  these  tasks,  AEDC  took  a  partial  B-1B  CFD  model  that  was  used  for  a 
previous  weapon  separation  evaluation,  modified  it  to  include  the  empennage,  and 
developed  flowfields  for  the  complete  aircraft.  Next,  AEDC  developed  a  flare  model  and 
evaluated  its  burning  characteristics.  The  flare  model  was  then  superimposed  onto  the  B- 
1B  flowfield  in  order  to  calculate  the  flare  trajectories. 

The  CTF  also  requested  support  from  NAWCWPNS  for  a  special  camera  system  to 
capture  the  flare  trajectories.  NAWCWPNS  specialists  used  their  recent  flare  testing 
experience  to  design  and  create  a  unique,  self-contained,  two-unit  camera  package  that 
was  built  inside  an  empty  flare  dispenser.  The  package  was  mounted  in  the  forward,  left- 
hand  dispenser  bay  of  the  B-1B  flight-test  aircraft.  A  narrow  field-of-view  (NFOV) 
camera  and  a  wide  field-of-view  (WFOV)  camera  were  mounted  to  provide  maximum 
film  coverage  of  the  flare  deployments. 

These  cameras  were  then  calibrated  to  provide  the  Y-Z  flare  position  information 
relative  to  the  aircraft.  In  addition,  a  NAWCWPNS  QF-4  was  equipped  with  an  Airborne 
Turret  Infrared  Measurement  System  III  (ATIMS  III)  and  flown  in  formation  with  the 
test  aircraft  to  capture  the  flare’s  X-position  data  relative  to  the  B-1B  flight-test  aircraft. 
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The  ATIMS  III  was  a  pod-mounted  instrument  and  data  acquisition  system  that  housed 
television  cameras,  IR  spectrometers  and  imaging  systems,  and  up  to  four  missile  seekers 
as  required  by  individual  test  customers.  The  flight-test  engineer  in  the  back  seat  of  the 
QF-4,  who  was  assisted  by  a  video  and  graphics  display  on  a  small  television  monitor, 
operated  the  turret.  During  post-test  analysis,  the  NFOV,  WFOV  and  ATIMS  images 
were  time-correlated  and  reduced  to  provide  center-of-flare  position  data  with  the 
extraordinary  accuracy  of  +1  ft. 

In  order  to  capture  the  flare  trajectory  data,  the  QF-4  flew  off  the  right  wing  of  the  B- 
1B,  at  co-altitude  and  positioned  back  at  the  4-5  o’clock  position.  The  B-1B  Defensive 
Systems  Officer  (DSO)  would  deploy  a  flare  at  the  specified  Mach  number  and  altitude. 
The  onboard  cameras  automatically  sequenced  to  the  “run”  position  with  the  firing  pulse. 
The  ATIMS  III  was  manually  activated  on  a  countdown  from  the  B-1B  DSO.  Flight-test 
conditions  varied  as  a  function  of  increasing  dynamic  pressure  from  0.70  Mach  (M)  and 
20,000  ft.  above  mean  sea  level  (MSL),  to  0.95  M  and  3,000  MSL.  The  original  flare  test 
plan  called  for  straight-and-level  points,  as  well  as  some  constant  bank  angle  points. 

However,  this  plan  was  altered  significantly  during  the  course  of  the  flight-test 
program.  Because  of  the  knowledge  of  the  flare  trajectories  gained  through  M&S, 
hazardous  flight  conditions  could  be  predicted  and  avoided.  This  was  a  significant  change 
in  the  test  team’s  approach  to  the  problem.  At  first,  the  team  was  collecting  data  in  order 
to  build  a  flare  trajectory  model.  However,  the  initial  model  was  very  accurate — within  2 
feet — that  the  model  was  used  to  determine  the  next  flight-test  point.  In  this  case,  M&S 
did  not  take  the  place  of  flight-test.  Flight-testing  was  essential  because  it  was  the  only 
way  to  collect  “truth  data”  for  the  M&S  effort.  However,  flight-te sting  could  not  be 
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accomplished  safely  independent  of  the  trajectory  model  provided  by  CFD  analysis.  In 
this  case,  M&S  were  essential  for  safely  accomplishing  the  test  objectives. 

Modeling  and  Simulation  Methods 

The  objective  of  M&S  was  to  investigate  the  cause  of  the  flare  strikes  on  the  B-1B 
empennage  by  modeling  the  flowfield  around  the  aircraft  and  then  by  evaluating  the 
effects  of  specific  aircraft  maneuvers  on  the  flowfield.  In  both  cases,  the  intent  was  to 
determine  if  the  flowfields  depressed  the  flare  trajectory  into  the  aircraft  and  then  to 
determine  the  specific  causes  of  lower  trajectories. 

The  original  M&S  plan  for  this  problem  was  to  develop  sets  of  flare  trajectories  for 
given  sets  of  aircraft  flight  conditions  and  flare  ejection  conditions.  The  tools  to 
accomplish  this  task  included  CFD  analysis,  engineering  methods,  and  solutions  to  the 
six  degree-of-freedom  (6-DOF)  equations  of  motion.  CFD  was  used  to  determine  the 
aircraft  flowfields.  Engineering  methods  were  used  to  develop  an  analytical  model  of  the 
aerodynamic  characteristics  of  the  flare.  The  flare  ejection  velocities  were  added  to  these 
models,  and  then  ah  of  these  models  were  used  as  inputs  to  the  6-DOF  equations  of 
motion  to  generate  the  flare  trajectories  in  the  presence  of  the  B-1B  flowfield. 

The  AEDC  team  began  by  modifying  a  previously  used  B-1B  CFD  model  created  for 
a  weapons  separation  program.  An  empennage  was  added  to  the  original  model,  which 
was  not  required  for  the  weapons  separation  analysis.  Input  modules  were  also  changed 
to  accept  CFD  initial  conditions  instead  of  wind  tunnel  inputs.11  The  trajectory  data  from 
the  NACWPNS  cameras  provide  the  initial  baseline  and  calibration  data  for  the  CFD 

Note 

11  Air  Force  Instruction  (AFI)  16-1001.  Verification,  Validation,  and  Accreditation 
(W&A),  draft.  This  draft  AFI  was  subsequently  published  1  June  1996. 
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model.  NACWPNS  data  was  also  used  to  validate  the  predictions  of  the  CFD  model  as 
the  dynamic  pressure  was  increased  in  a  build  up  approach  during  the  flight-test. 

This  CFD  model  used  a  computational  approach  known  as  the  chimera  overset  mesh 
methodology.  The  term  “chimera”  applied  to  the  scheme  because  of  its  similarities  with 
the  Roman  mythological  creature  that  was  created  from  parts  of  several  other  animals. 
Like  its  mythological  namesake,  the  chimera  scheme  took  component  flowfield  grids 
around  the  aircraft,  such  as  the  wings,  fuselage,  nacelles,  empennage  and  then  connected 
them  into  a  system  of  relatively  simple  overlapping  computational  meshes  to  construct  a 
complete  model.  In  this  case  a  total  of  22  meshes,  and  over  1.1  million  total  mesh  points 
were  generated  to  cover  half  of  the  aircraft,  and  then  the  principle  of  symmetry  was  used 
to  cover  the  other  half  (see  Figure  4:  B-1B  Computational  Mesh  System).  As  the  CFD 
flowfield  solution  was  initiated  in  the  computer,  the  boundary  conditions  for  each  point 
downstream  in  the  meshes  were  interpolated  from  the  flow  properties  in  the  overlapping 
meshes. 
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Figure  4:  B-1B  Computational  Mesh  System 
(Meshes  Coarsened  for  Clarity) 

Three  significant,  simplifying  assumptions  were  also  made  during  this  early  phase  of 
the  M&S  effort.  First,  the  assumption  of  inviscid  flow  was  made,  which  considerably 
simplified  the  problem.  This  assumption  was  based  on  the  results  of  a  previous  weapons 
separation  analysis,  which  had  originally  used  fully  viscous  flow  model.  The  previous 
analysis  indicated  there  were  no  significant  viscosity  effects  over  the  top  of  the  aircraft. 
Therefore,  the  coefficient  of  viscosity  set  to  zero  in  the  computer  model  and  the 
simulations  were  run  again  as  inviscid  (Euler)  equations,  which  yielded  similar  results  to 
the  fully  viscous  Navier-Stokes  equations.  The  second  simplifying  assumption  was 
flowfield  symmetry  about  the  X-Z  (vertical)  plane  of  the  aircraft  was  assumed  for  cases 


involving  no  sideslip  or  roll  rate.  For  those  cases  with  roll  rate  or  sideslip,  the  full  aircraft 
model  was  used  to  account  for  the  cross-flow  components. 

The  third  modeling  assumption  was  to  model  the  flare  as  a  cylinder  in  the  flowfield. 
In  order  to  validate  this  assumption,  the  first  consideration  was  to  determine  whether  the 
burning  of  the  flare  contributed  significantly  to  the  trajectory  of  the  flare,  either  from  loss 
of  mass  or  from  changing  shape.  Tracor  provided  test  data,  which  indicated  that 
approximately  3  percent  of  the  flare  mass  was  consumed  during  the  0.3  to  0.5  seconds 
required  for  the  flare  to  pass  the  rear  plane  of  the  vertical  tail.12  Because  such  a  small 
portion  of  the  flare  consumed  during  this  period,  mass  flux  could  not  significantly  affect 
the  flare  trajectory  either.  Once  the  non-buming  flare  model  was  assumed,  the  flare 
trajectory  problem  became  a  “classic”  cylinder  in  a  fluid  stream  described  in  most 
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Figure  5:  Flight  Conditions  Modeled 


Note 

12 

Mr.  Neal  Brune,  Tracor  Corporation,  to  B-1B  Sustainment  Office,  letter,  subject:  MJU- 
23B  Defensive  Flare  Technical  Specifications,  1  November  1996. 
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1  T 

textbooks  on  fluid  dynamics  (see  Figure  6:  Model  of  the  MJU-23/B  Flare  Pellet). 

AEDC  modeled  the  flare  using  a  semi-empirical  aerodynamic  coefficient  prediction 
program  that  had  been  generated  for  other  projects — primarily  weapons  separation 
evaluations.  This  flare  model  was  then  placed  into  the  aircraft  flowfield  model.  At  this 
point,  the  modeling  effort  was  complete  and  the  computer  began  computing  flare 
trajectories  for  simulated  flight  conditions. 

In  total,  the  AEDC  team  developed  seventeen  different  flowfields.  The  primary 
variables  in  these  flowfields  were  angle-of-attack,  sideslip,  mach,  and  roll  rate.  These 
flowfield  models  were  then  used  to  generate  over  700  different  flare  trajectories  (see 
Figure  5:  Flight-Test  Conditions  Modeled).  Once  these  trajectories  were  validated  by 
flight-test,  a  linear  Taylor  series  model  was  developed.  The  Taylor  series  model  defined 
the  point  in  space  where  the  flare  would  pass  the  trailing  edge  of  the  tail  for  a  given  set  of 
flight  parameters.  The  Taylor  series  model  tremendously  reduced  the  computer  resources 
required  but  was  only  useful  because  it  had  been  validated  with  the  full  CFD  solutions 
which  had  in  turn  been  validated  by  flight-test. 

As  the  completed  computational  model  developed  solutions,  these  solutions  were 
compared  against  the  results  that  had  been  generated  during  the  early  portion  of  the 
flight-test  effort.  The  success  of  the  M&S  effort  was  immediately  apparent.  The  statistical 
variance  between  the  actual  flight-test  results  and  the  model  results  for  a  given  set  of 
ejection  velocities  and  flight  conditions  were  negligible  for  both  straight  and  level  flight 
and  steady  banked  flight.  One  important  conclusion  became  immediately  apparent  from 
the  results  at  this  early  stage.  As  long  as  the  flare  ejection  velocity  was  within  its 

Note 

13  John  D.  Anderson,  Introduction  to  Flight ,  3rd  ed.  (New  York:  McGraw-Hill,  1989), 
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specified  design  conditions,  the  flares  could  not  physically  hit  the  tail  of  the  aircraft 


during  straight  and  level  flight  or  steady  banked  flight  (see  Figure  7). 14 
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Figure  6:  Model  of  MJU-23/B  Flare  Pellet 
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Figure  7:  Comparison  of  Flight-test  Data  with  Computational  Model 
for  Straight  and  Level  Flight 

Notes 

133-135. 

14  Major  John  H.  Humphries  and  Major  David  E.  Miller,  “B-1B/MJU-23B  Flare  Strike 
Test  Program,”  Proceedings  of  the  40th  Symposium  of  the  Society  of  Experimental  Test 
Pilots,  September  1996 
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Engineering  Results 


The  revelation  of  no  flare  strikes  in  level  flight  led  to  a  complete  revision  of  the 
flight-test  plan  to  include  various  maneuvers.15  Since  the  number  and  style  of  defensive 
maneuvers  was  practically  unlimited,  choosing  “operationally  representative”  maneuvers 
became  the  next  issue.  Fully  repeatable  maneuvers,  including  rolls  and  bunts,  were 
required  and  devised  to  validate  the  maneuvering  models  at  AEDC.  Both  deterministic 
(flare  trajectory)  and  stochastic  (random  maneuver)  data  were  required.  In  addition,  the 
CTF  began  to  record  a  series  of  “gun-jinks”  to  provide  insights  into  the  transient 
flowfield  characteristics  during  defensive  maneuvers. 

The  M&S  effort  resulted  in  increased  test  efficiency  during  this  phase  of  the 
program.  The  CFD  models  permitted  the  team  to  use  a  “predict-fly-analyze”  approach, 
instead  of  a  “fly-fix -fly”  approach.  The  AEDC  team  analyzed  these  maneuvers  for  angle  - 
of-attack,  roll,  and  airspeed  content  to  create  similar  maneuvers  in  the  computer  model. 
Flares  were  deployed  during  rolling  maneuvers  using  the  procedures  in  the  initial  flight- 
test  program. 

However,  this  effort  was  cut  abruptly  short  when  the  QF-4  chase  plane  could  not 
visually  verify  that  the  flares  were  not  actually  striking  the  tail  during  the  roll.  On  four 
out  of  twelve  events,  the  chase  plane  had  to  rejoin  on  the  B-1B  to  investigate  apparent 
strikes.16  Although  none  of  the  flares  actually  struck  the  aircraft,  the  flight-test  team 
readily  recognized  that  continuing  the  tests  would  be  hazardous.  Therefore,  flight-tests 

Note 

15  419  Flight  Test  Squadron,  “MJU-23B  Flare  Trajectory  Analysis  Test  Plan,”  Edwards 
AFB,  CAJVlarch  1995.  33. 
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were  suspended  until  M&S  could  provide  some  critical  answers.  In  the  meantime,  AEDC 
agreed  to  continue  M&S  work  with  the  rolls  and  other  maneuvers  using  the  data  points 
already  obtained  to  validate  the  models. 

In  order  to  handle  the  maneuvering  flight  requirement  in  the  computer  model,  several 
tools  at  AEDC  had  to  be  modified  and  then  applied.  The  first  of  these  tools  was  a  two- 
body  motion  program  originally  developed  for  the  F-22  program  to  investigate  the  effects 
of  a  rolling  aircraft  launching  a  missile.  This  model  was  modified  to  accept  CFD  inputs 
instead  of  wind  tunnel  inputs  and  the  respective  aircraft  and  “weapon”  model  changes 
were  made.  The  second  tool  was  a  modeling  scheme  called  “dynamic  flowfield 
interpolation”  that  was  developed  at  AEDC  for  an  F-15E  weapons  separation  program. 
This  technique,  which  was  validated  with  flight-test  results,  allowed  the  modeler  to  add 
dynamic  flight  parameters  on  top  of  a  constant  aircraft  flowfield  to  generate  a 
‘  ‘maneuvering’  ’  aircraft . 1 7 


Notes 

16  Naval  Air  Warfare  Center  Point  Mugu.  Flight  Tests  of  MJU-23/B  Flares  (U).  In-house 
production.  40  min.,  1996,  videocassette. 

17 

Air  Force  Instruction  (AFI)  16-1001.  Verification,  Validation,  and  Accreditation 
(W&A),  draft 
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Figure  8:  Comparison  of  Flight-test  Data  with  Computational  Model  for  a  Rolling 

Aircraft  Maneuver 

In  particular,  the  algorithm  would  use  the  time-dependent  flight  parameters  (e.g., 
angle-of-attack,  sideslip,  roll  rate,  and  normal  acceleration)  to  “bend”  the  vectors  of  the 
constant  angle-of-attack  aircraft  flowfield.  These  two  tools,  employed  with  standard 
engineering  methods  and  “rolling”  aircraft  data  from  the  Rockwell’s  B-1B  simulator, 
allowed  the  AEDC  team  to  successfully  model  the  flares  during  the  rolling  maneuvers 
prescribed  in  the  flight-test  plan. 

The  model  predictions  for  40  and  60  degrees/second  roll  rates  were  validated  by  the 
CTF  during  the  flight-test  (see  Figure  8).  These  analyses  indicated  that  the  flares  could 
strike  the  tail  of  the  aircraft  for  any  flight  condition  above  0.85  M  when  the  flare  ejection 
velocity  was  80-fps  (i.e.,  the  bottom  of  the  ejection  velocity  spec).  At  100-fps  ejection 


25 


velocity,  the  analysis  indicated  that  B-1B  tactical  maneuvers  did  not  cause  the  flare  pellet 
to  strike  the  tail,  even  at  negative  angles-of-attack  (negative  angle-of-attack  “pushed”  the 
tail  up  toward  the  trajectory  of  the  flare).  This  result  was  depicted  as  a  “hurdle”  which  the 
flare  could  clear,  regardless  of  roll  rate  or  sideslip,  at  the  worst-case  negative  angle-of- 
attack  (see  Figure  9). 18 

Another  important  part  of  the  analysis  effort  was  to  consider  the  flare  ejection 
velocities  from  lot  acceptance  testing  data  provided  by  Tracor.  This  part  would  allow  the 
team  to  determine  if  the  80-fps  flare  was  a  practical  threat.  The  existing  velocity  data, 
consisting  of  168  flare  ejection  tests,  reflected  a  normal  distribution  and  could  be 
analyzed  using  “textbook”  statistical  procedures.19  This  provided  the  important  insight 
that  an  80-fps  ejection  velocity  was  three  standard  deviations  below  the  mean.  This  meant 
that  the  chance  of  an  80-fps  ejection  was  about  one  in  one  thousand.  The  average  ejection 
velocity  was  slightly  more  than  100-fps. 

The  contributions  of  the  modeling  effort  were  evident  throughout  the  test  process. 
With  validated  models,  AEDC  could  then  apply  the  Taylor-series  expansion  model  to 
generate  solutions  quickly  for  the  point  at  which  the  flare  passed  the  plane  of  the  vertical 
tail.  These  solutions  were  then  used  to  create  the  safe  training  envelope  required  by  Air 
Combat  Command  (see  Appendix  A).  The  validation  of  the  Taylor  series  solutions 
came  from  the  AEDC  CFD  model  and  flight-test  data.  Note  that  the  terms  “Safe”  and 

Note 

Arnold  Engineering  Development  Center,  B-1B  Flare  Strike  Investigation ,  In-house 
production,  22  min.,  1996.  videocassette. 

19  Lawrence  L.  Lapin,  Quantitative  Methods,  3rd  ed.  (San  Diego:  Harcourt  Brace 
Jovanovich,  1985),  62-65. 

“  Air  Force  Instruction  (AFI)  16-1001.  Verification,  Validation,  and  Accreditation 
(W&A),  1  June  1996. 
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“Unsafe”  in  the  results  depicted  in  Appendix  A  refer  to  whether  or  not  the  flare  would 
penetrate  the  3-foot  box  that  was  established  by  OC-ALC  as  the  desired  safety  margin.  In 
the  case  of  the  0.95  Mach  envelope,  for  example,  all  flares  cleared  the  top  of  the  tail,  but 
the  “hurdle”  was  less  than  3  feet  above  the  top  of  the  tail. 

With  the  modeling  validated  by  flight-test  data,  the  team  made  several  conclusions. 
First,  the  team  recommended  that  ACC/DO  expedite  the  replacement  of  ejection 
cartridges  to  ensure  the  100-fps  minimum  ejection  velocity  flares  entered  service.  Until 
that  point,  all  flare  deployments  above  0.85  Mach  should  be  restricted  to  straight  and 
level  flight.  Second,  after  the  new  flares  and  cartridges  were  installed,  the  team 
recommended  that  all  restrictions  be  removed.  The  team  stressed  that  the  minor  damage 
would  continue  until  the  new  flare  design,  which  minimized  the  use  of  metal  sub- 
assemblies,  could  be  fielded.  Even  at  that  point,  there  could  be  no  guarantee  that  the  fleet 
would  not  see  an  occasional  “dud”  flare  (i.e.,  one  that  failed  to  reached  minimum 
specification  ejection  velocity). 


Notes 

21 

“  Mr.  Neal  Brune,  Tracor  Corporation,  to  B-1B  Sustainment  Office,  letter,  subject:  MJU- 
23B  Defensive  Flare  Technical  Specifications,  1  November  1996. 
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Figure  9:  Vertical  Displacement  for  Flare  with  100-fps  Ejection  Velocity  during  Negative 

Angle-of- Attack  Maneuvers 

Evaluating  M&S  for  Flight-Testing  Physical  Components 

In  this  case,  M&S  allowed  the  test  team  to  solve  a  very  difficult  engineering  problem 
at  the  physical  component  level.  The  flare  strikes  were  also  a  hazard  to  flight  safety. 
Without  the  predictive  power  of  M&S  technology,  the  test  team  would  have  been  forced 
to  fly  many  test  points  looking  for  a  seemingly  “random”  flare  strike  in  order  to  begin 
looking  for  a  solution.  Even  after  determining  the  general  flight  conditions  where  flare 
strikes  might  happen,  it  would  be  impossible  to  develop  a  solution  without  taking  a 
statistically  significant  sample  of  low  flare  ejection  velocities  at  the  appropriate 
conditions. 

Therefore,  statistically  significant  samples  of  two  events  would  have  been  required. 
First,  flight-testers  must  find  the  maneuvering  flight  conditions  that  “push”  the 
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empennage  closer  to  the  flare  trajectory.  Second,  a  statistically  significant  sample  of  low 
ejection  velocities  was  required  in  order  to  assess  the  risks  of  a  flare  strike. 

This  approach  would  have  required  many  high  risk  flight-test  sorties  to  achieve  the 
same  engineering  results  that  six  low  risk  flight-test  sorties  accomplished  with  the  benefit 
of  M&S  technology.  In  this  case,  a  cost-benefit  analysis  is  straightforward.  The  B-1B 
costs  $35,000  per  hour  to  flight-test.  The  flare  strike  program  required  twenty  hours  of 
flight-time  for  a  total  cost  of  $700,000.  The  M&S  effort  cost  the  Air  Force  less  than 
$200,000.  Total  program  costs  were  $950,000  with  including  program  and  miscellaneous 
management  costs  figured  in.  Without  the  benefit  of  M&S  technologies,  at  least  one 
hundred  B-1B  flight-test  hours  would  have  been  required  to  get  the  same  results.  This 
would  have  resulted  in  $3.5  million  in  program  costs  and  taken  more  than  a  year  to  fly. 
Therefore,  M&S  was  a  highly  cost  effective  tool  for  modeling  physical  components. 

The  second  criterion  for  evaluating  M&S  is  engineering  validity.  Fortunately,  the 
flight-test  instrumentation  that  was  especially  designed  for  this  project  was  extremely 
accurate.  The  combination  of  cameras  in  the  flare  canisters  and  the  ATIMS  III  on  the 
chase  aircraft  allowed  engineers  to  determine  the  flare  position  within  +  1  foot  along  its 
trajectory.  This  served  as  “truth”  data  forjudging  the  engineering  validity  of  the  CFD  and 
Taylor  series  models  of  the  trajectories.  In  this  case,  every  CFD  solution  and 
corresponding  flight-test  data  point  were  within  2  feet  of  one  another.  The  M&S  results 
were  valid  and  sufficiently  accurate  to  provide  ACC  with  the  safe  training  envelope. 

The  last  criterion  for  evaluating  the  M&S  effort  on  the  flare  strike  program  is 
comprehensiveness.  At  the  component  level,  the  model  was  comprehensive.  There  were 
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no  surprises  during  flight-test,  because  flight-test  data  matched  the  M&S  predictions  so 
well.  There  were  no  anomalies  or  unexpected  results. 

M&S  solutions  did  not  replace  flight-test.  Instead,  M&S  allowed  flight-testers  to 
attack  more  difficult  problems.  Also,  M&S  greatly  enhanced  the  engineering  analysis  of 
flight-test  data.  M&S  methods  were  cost  effective,  valid,  and  comprehensive.  Ultimately, 
the  flight-test  results  from  this  program  were  afforded  only  through  the  strengths  of  M&S 
tools  and  the  computational  techniques  required  to  run  them.  M&S  were  also  essential  to 
the  engineering  flare  redesign  effort  and  getting  the  fleet  back  to  full  operational  status. 
These  are  rich  dividends  for  small  investments  in  M&S  capabilities! 
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Chapter  3 


Case  Study  Two  -  Control  Systems  Modeling  with  the  Variable 
Stability  In-Flight  Test  Aircraft 


We  should  never  test  without  simulating  first. 

Honorable  Paul  Kaminski 
Undersecretary  of  Defense 

Introduction:  Why  do  simulations  in-flight? 

In-flight  simulation  has  proven  to  be  a  useful,  cost-effective  tool  that  allows  test 
pilots  to  go  beyond  the  limited  capabilities  of  ground-based  simulations  and  into  the 
actual  flight  environment.  An  in-flight  simulator  is  an  aircraft  that  can  simulate  the 
handling  qualities  and  flying  characteristics  of  another  aircraft  in-flight.  Aircraft  designed 
to  serve  as  in-flight  simulators  use  especially  designed  aerodynamic  controls  along  with 
accurate  model-following  control  input  systems  to  permit  the  in-flight  simulation  aircraft 
to  produce  motions  that  duplicate  the  responses  of  another  aircraft  in-flight. 

The  in-flight  simulator  uses  standard  M&S  techniques  to  predict  the  responses  of  the 
simulated  aircraft — just  like  a  ground-based  simulator.  However  unlike  ground-based 
simulators,  an  in-flight  simulator  has  total  control  over  all  six  degrees-of-freedom  for  a 
rigid  body.  Therefore,  an  in-flight  simulation  produces  the  complete  kinesthetic  and 
visual  environment  of  flight  rather  than  the  limited  and  synthetic  cues  of  a  ground 
simulation.  Because  the  variable  stability  in-flight  simulator  aircraft — also  known  as  the 
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VISTA — is  an  actual  aircraft,  it  provides  a  degree  of  realism  not  present  in  ground 
simulation.  The  pilot  experiences  the  real  flight  motions,  accelerations,  and  handling 
qualities  of  the  simulated  aircraft,  in  actual  flight  conditions.  This  realism  gives  the  pilot 
a  higher  level  of  confidence  in  the  simulation  results. 

In  addition  to  the  important  advantages  of  providing  more  realistic  environment,  in¬ 
flight  simulation  provides  the  Air  Force  with  an  extraordinarily  flexible  tool  to  help 
reduce  the  cost  and  schedule  of  developmental  test  and  engineering  work.  Indeed,  the  two 
most  important  advantages  of  in-flight  simulation  are  that  it  lowers  both  the  costs  and 
risks  of  testing  flight  control  systems,  (see  Figure  10) 

In-flight  simulation  also  offers  greater  flight  test  efficiency  and  flexibility.  Because 
the  simulation  is  controlled  by  software  that  can  be  modified  or  changed  in-flight,  the 
flight-test  team  has  the  ability  to  fly  several  simulations  using  one  airframe  and  one 
sortie.  The  in-flight  simulator  also  has  the  advantage  of  being  easily  modified  for  future 
growth.  By  simply  changing  the  flight  simulation  computers  and  software,  the  system  is 
easy  to  upgrade.22 


VISTA 


Note 

22  Philip  A.  Reynolds,  VISTA/F-16  Design  Analysis  Report,  CAL-7693-7  (Wright- 
Patterson  AFB,  OH.:  Air  Force  Material  Command,  24  January  1990), 15. 
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In-flight  simulation  also  allows  engineers  to  gather  the  data  necessary  to  make 
scientific  predictions  of  aircraft  systems  performance.  Because  VISTA  simulations  are 
software  driven,  hardware  prototypes  are  not  required.  A  prototype  aircraft  can  be 
extraordinarily  expensive  to  build  and  modify,  and  the  loss  of  a  single  prototype  could 
put  an  Air  Force  acquisition  program  years  behind  schedule.  The  Air  Force  simply  cannot 
afford  to  risk  a  prototype  aircraft  in  order  to  troubleshoot  aircraft  systems  and  software 
when  an  in-flight  simulation  can  do  the  job  less  expensively  and  more  safely. 

Historical  Background  of  the  VISTA 

The  original  in-flight  simulator — the  venerable  NT-33 — was  a  T-33  airframe  fitted 
an  analog  variable  stability  system  and  a  rudimentary  variable  feel  control  system.  It  was 
upgraded  many  times  and  served  the  Air  Force  well  from  1965  to  1995.  During  its  long 
and  distinguished  career,  the  NT-33  provided  important  flying  qualities  data  and  response 
information  for  many  newer  aircraft.  In  fact,  the  NT-33  was  used  to  develop  and  refine 
the  Air  Force’s  first  fly-by- wire  control  system  on  the  F-16.  Additionally,  Mil-Std 
1797A  Flying  Qualities  of  Piloted  Aircraft,  is  filled  with  data  collected  by  in-flight 
simulation  in  the  NT- 3 3  aircraft. 

The  VISTA  or  Variable  Stability  In-flight  Simulator  Test  Aircraft  was  designed  to 


Figure  10:  Benefits  of  In-flight  Simulation 
(courtesy  of  CALSPAN  Corp.) 

Note 

23  Major  Robert  A.  Wilson  and  Mr.  Karl  T.  Hutchinson,  "What  is  VISTA?  An 
Introduction  to  Variable  Stability  In-Flight  Simulator  Test  Aircraft,"  Proceeding  of  the 
38th  Annual  Symposium  of  the  Society  of  Experimental  Test  Pilots,  1  September  1994,  78. 
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replace  the  aging  variable  stability  NT-33.  Because  of  the  void  created  by  the  retirement 
of  the  NT-33,  the  Air  Force  tasked  Calspan  Corporation,  the  previous  operators  of  the 
NT-33,  to  define  the  requirements  for  the  next  generation,  high  performance  in-flight 
simulator.  Calspan  identified  four  specific  requirements,  which  were  validated  by  Air 

24 

Force  Materiel  Command:" 

1.  The  airplane  must  be  a  two-seat  fighter  type  aircraft  with  variable  feel  controls 
and  programmable  displays  in  the  front  cockpit  and  all  pilot-in-command 
functions  in  the  rear  cockpit. 

2.  The  Variable  Stability  System  (VSS)  must  be  capable  of  all  attitude  model¬ 
following  control  and  response  feedback  simulations. 

3.  The  aircraft  should  have  control  of  six  degree-of-freedom  forces  and  moments  to 
satisfy  mission  and  simulation  fidelity  requirements. 

4.  It  should  be  relatively  inexpensive  to  operate. 

In  order  to  meet  these  requirements,  a  modem  fighter  type  aircraft  would  be  required 
to  meet  the  simulation  and  performance  specifications.  Additionally,  the  aircraft  chosen 
to  become  the  next  generation  in-flight  simulator  should  be  easy  to  retrofit  with  a  fly-by- 
wire  variable  stability  control  system.  Finally,  cost  was  a  major  program  concern.  Both 
the  Air  Force  and  Calspan  Corporation  hoped  to  use  an  aircraft  that  was  already  in  the 
inventory.25 

Several  fighter  type  aircraft  were  considered,  but  the  F-16D  was  considered  the  most 
suitable  because  the  digital  flight  control  system  on  the  production  aircraft  was  most  the 

Note 

24  Ibid.  79. 

“  Program  Management  Directive  for  Variable  Stability  In-Flight  Test  Aircraft  (VISTA), 
21  March  1991. 
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easily  retrofitted  for  variable  stability  control  system.26  Headquarters  Air  Combat 
Command  and  Headquarters  Air  Force  supported  this  decision  by  assigning  a  production 
F-16D  to  the  in-flight  simulator  program  at  no  cost  to  the  program.”  Air  Force  Materiel 
Command  approved  Calspan’s  engineering  proposals  and  Wright  Laboratories  awarded 
the  modification  contract  shortly  thereafter. 

VISTA  Aircraft  Description 

The  VISTA  was  a  Block  30  F-16D  with  the  Israeli  Peace  Marble  II  dorsal  spine.  The 
dorsal  spine  was  added  to  the  basic  aircraft  configuration  to  house  the  computers  used  by 
the  variable  stability  system  and  the  aircraft’s  data  recording  system.  The  airplane  was 
equipped  with  a  digital  flight  control  computer  and  was  powered  by  a  General  Electric 
FI  10-100  engine.  The  addition  of  a  Variable  Stability  System  (VSS)  gives  the  airplane  its 
in-flight  simulation  capability.  Extensive  cockpit  modifications  were  made  to  place  the 
pilot  in  the  command  station  in  the  rear  cockpit.  Additionally,  an  airborne  test 
instrumentation  system  (ATIS)  was  installed  providing  on-board  data  recording  and  real 
time  telemetry  capability. 

Sideslip  sensors —  called  beta  cones — similar  to  production  F-16  angle  of  attack 
cones  were  mounted  on  the  top  and  bottom  of  the  forward  fuselage.  A  flight  test  nose 
boom  was  also  installed  for  the  developmental  test  program  to  calibrate  the  pitot- static 


Note 

26  David  M.  North,  “VISTA  Primed  for  Research,  Training,”  Aviation  Week  and  Space 
Technology  142  no.  10  (Mar  6,  1995):  54-56. 

27  Wilson  and  Hutchinson,  80. 
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system  of  the  aircraft.  Several  other  internal  modifications  were  made  to  support  the 
additional  demands  made  by  the  variable  stability  flight  control  system.” 

These  changes  included  the  addition  of  F-16XL  51 -gallon  per  minute  hydraulic 
pumps  instead  of  the  standard  42  gallons  per  minute  pumps  and  the  removal  of  hydraulic 
flow  restrictors  in  the  servo-actuators  for  the  flaperons  and  horizontal  tails.”  The  primary 
purpose  of  the  hydraulic  modifications  was  to  increase  hydraulic  system  capacity  to 
support  the  rapid  control  actuations  required  for  in-flight  simulation.  A  more  robust 
hydraulic  system  also  allowed  for  future  upgrades  to  the  variable  stability  control  system. 

Perhaps  contrasting  the  two  cockpits  in  the  VISTA  with  a  production  F-16D  best 
illustrates  the  uniqueness  of  the  VISTA  aircraft.  The  most  obvious  change  to  the  front 
cockpit  was  the  addition  of  a  movable  variable-feel  center  stick.  In  order  to  make  room 
for  the  center  stick,  the  primary  flight  instruments  in  the  front  cockpit  were  removed.  All 
pilot-in-command  functions  were  removed  from  the  front  cockpit  making  the  instrument 
panel  look  almost  exactly  like  the  rear  cockpit  of  a  production  F-16D.  A  removable  hard 
drive  similar  to  a  production  Data  Transfer  Unit  Cartridge  (DTUC)  was  installed  in  the 
front  cockpit  that  interfaced  with  the  variable  stability  system  computers.  This  hard  drive 
contained  as  many  as  200  pre-programmed  flight  control  configurations  and  could  be 

30 

removed  and  replaced  in-flight  by  the  front  cockpit  pilot. 

Another  major  feature  of  the  front  cockpit  was  the  variable  feel  control  system.  Stick 
force  and  position  requirements  for  the  center  stick  could  be  changed  to  simulate  various 

Note 

28  Karl  T.  Hutchinson,  ed..  Maintenance  Manual  for  the  VISTA/F-16  Variable  Stability 
Aircraft,  CAL-7693-10  (Buffalo,  NY.  January  1992) 

“  Wilson  and  Hutchinson,  81. 

30  Karl  T.  Hutchinson,  ed.,  VISTA/F-16  Variable  Stability  System  Pilots’  Operating 
Manual,  CAL-7693-9  (Buffalo,  NY.  January  1992),  1-38. 
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aircraft  configurations  and  flight  conditions.  Because  the  side  stick  in  the  front  cockpit 
was  the  standard  F-16  side  stick  with  only  lA  inch  of  motion  available — only  force 
command  configurations  were  available  through  the  VSS  with  this  stick.  The  variable 
feel  system  could  also  vary  other  parameters  such  as  trim  pre-loads,  breakout,  friction, 
nonlinear  force  gradients,  and  bob-weights  on  the  controls.  Changes  in  stick  force 
gradients,  friction,  and  breakout  forces  could  all  be  independently  programmed  with  the 
variable  feel  system. 

A  less  obvious  change  to  the  front  cockpit  was  that  the  throttle  was  not  mechanically 
connected  to  the  rear  cockpit  throttle  or  directly  to  the  engine  controls.  Instead,  the 
throttle  in  the  front  cockpit  of  the  VISTA  worked  entirely  through  the  variable  stability 

o  1 

system.  Because  the  throttle  was  not  mechanically  connected  to  the  engine  and  the  fuel 
shutoff  switch  had  been  removed,  the  engine  could  not  be  started  or  shut  down  from  the 
front  cockpit. 

The  rear  cockpit  was  very  similar  to  the  appearance  and  function  of  the  front  seat  of 
a  production  F-16.  All  pilot-in-command  functions  were  relocated  from  the  front  to  the 
rear  cockpit — including  engine  start  and  shut  down  controls.  The  front  and  rear  throttles 
were  not  mechanically  connected.  This  design  feature  allowed  the  engine  to  be  used  as 
part  of  the  simulation.  When  the  VSS  was  engaged,  the  computers  monitored  the  front 
cockpit  throttle  position  and  commanded  engine  response  according  to  simulation 
requirements.  Although  the  rear  cockpit  throttle  did  not  send  inputs  to  the  engine  with  the 
VSS  engaged,  the  rear  cockpit  throttle  position  was  driven  by  a  servo  to  match  the  VSS 
engine  command.  This  minimized  thrust  transients  when  the  VSS  was  disengaged. 

Note 

31  ibid.  1-56. 
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Figure  11:  The  VISTA  Aircraft 


Modeling  and  Simulation  Methods 


VISTA  Variable  Stability  System 

The  Variable  Stability  System  (VSS)  in  the  VISTA  aircraft  provided  limited  six 
degrees  of  freedom  simulation  capabilities.  The  system  commanded  symmetric  and 
asymmetric  horizontal  tail  movements,  symmetric  and  asymmetric  flaperon  movements, 
rudder  movements,  and  throttle  control.  The  only  surfaces  not  controlled  by  the  VSS 
were  the  leading  edge  flaps  and  speedbrakes.  The  flaperons  could  be  positioned 
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symmetrically  both  trailing  edge  up  and  trailing  edge  down  to  provide  direct  lift  control. 
The  brains  of  the  VSS  system  were  three  flight  control  computers  salvaged  from  Hawk 
missile  systems  that  calculated  aircraft  control  surface  actuation  necessary  to  perform  the 
in-flight  simulation.  An  additional  flight  control  computer  from  a  Titan  missile  performed 
the  function  of  controlling  the  variable  feel  system  on  the  center  stick. 

The  rear  cockpit  of  the  VISTA  did  not  incorporate  the  VSS  flight  controls  and  was 
always  configured  as  a  production  F-16D  aircraft.  The  rear  cockpit  only  had  the  standard 
F-16  side  stick  controller  and  used  the  production  software  to  drive  the  flight  controls 
exclusively.  The  F-16  flight  control  computers  and  software  were  isolated  from  the  VSS 
system.  The  VSS  was  also  designed  to  degrade  gracefully  to  the  F-16D  production  flight 
control  laws.  However,  the  control  panel  in  the  rear  cockpit  could  manually  “lock-out” 
the  VSS  signals  to  the  control  surfaces  in  case  of  an  emergency.34 

The  VSS  in  the  front  cockpit  of  the  VISTA  was  functional  in  one  of  three  separate 
modes:  variable  stability,  F-16  emergency  mode,  and  F-16  convenience.  A  major  point 
for  the  flight  crew  to  remember  is  that  in  order  to  preserve  the  simulation  algorithms  used 
for  variable  stability,  only  one  cockpit  could  be  active  at  a  time  when  the  VSS  was 
driving  the  flight  controls.  ~  When  the  rear  cockpit  pilot  was  flying,  the  front  cockpit 
controls  were  inoperative  and  vice  versa  when  the  front  cockpit  was  flying,  the  rear 
cockpit  controls  were  inoperable.  The  only  time  both  cockpits  are  active  simultaneously 
is  when  the  backup  emergency  mode  is  used.  This  backup  mode  is  simply  an  independent 

Note 

'  Wilson  and  Hutchinson,  81. 

33  ibid.  82. 

14  Karl  T.  Hutchinson,  ed.,  VISTA/F-16  Variable  Stability  System  Pilots’  Operating 
Manual,  CAL-7693-9  (Buffalo,  NY.  January  1992),  3-10. 

35  ibid.  2-15. 
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flight  control  system  that  uses  normal  production  F-16D  flight  control  computers  and 
software.  This  independent  back-up  system  is  available  in  case  of  a  VSS  malfunction.  It 
provides  a  way  for  the  front  seat  pilot  to  fly  without  the  VSS  in  an  emergency  and  cannot 
interfere  with  the  normal  flight  control  system.  This  provided  another  layer  of 
redundancy  and  safety  in  case  of  computer  malfunctions. 

The  variable  stability  system  was  used  for  all  simulation  work.  The  F-16  emergency 
mode  did  allow  the  front  cockpit  pilot  to  fly  the  aircraft  using  the  side  stick  and  normal  F- 
16  control  laws  through  the  VSS.  This  also  allows  the  front  seat  pilot  an  emergency 
means  to  recover  the  aircraft  should  the  rear  cockpit  safety  pilot  be  unable  to  do  so.  The 
F-16  convenience  mode  is  very  similar  to  the  emergency  mode.  Convenience  allows  the 
front  seat  pilot  to  fly  the  aircraft  while  the  rear  seat  pilot  tends  to  other  duties  such  as 
reprogramming  the  VSS  computers  or  recording  flight  test  data.  The  main  difference 
between  convenience  and  emergency  mode  is  that  in  convenience,  all  manual  safety  trips 
that  revert  control  to  the  rear  cockpit  are  still  available  and  the  rear  cockpit  pilot 
maintains  throttle  control  if  desired.  Figure  12  below  summarizes  the  various  flight 
control  modes. 


VISTA  Flight  Control  Modes 


Front  Cockpit 

Rear  Cockpit 

Variable  Stability 

Always  F-16  flying  qualities 

F-16  Emergency 

F-16  Convenience 
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Notes:  Only  one  cockpit  may  be  active  at  a  time. 
The  back  up  emergency  mode  is  separate  from  the  VSS  and  provides  a 
redundant  flight  control  system. 


Figure  12:  Flight  Control  Modes 

Engineering  Results  Flight-Testing  Robust  Flight  Controls  for  the  F-22 

All  aircraft  depend  on  their  flight  control  systems  for  safe,  satisfactory  operation. 
Assuring  stability,  trim,  and  performance  with  all  components  functioning  properly  is  the 
primary  design  goal.  Stability,  trim,  and  performance  must  also  be  maintained  following 
digital  system  failures.  Digital  fly-by- wire  flight  control  failures  also  have  entirely 
different  failure  characteristics  than  mechanical  flight  control  systems.  Although 
computer  software  seldom  fails,  it  is  susceptible  to  latent  programming  faults  that  may 
surface  unexpectedly  and  may  be  hard  to  detect.  Unfortunately,  these  failures  are  seldom 
subtle  or  graceful.  Multiple  failures  can  also  occur,  particularly  because  of  physical 
damage.  While  not  strictly  systems  failures — pilot  error,  power  transients,  and 
meteorological  hazards — can  produce  system  states  that  require  prompt  responses.36 
Therefore,  control  system  logic  must  do  more  than  just  accommodate  isolated  single¬ 
point  failures.  The  VISTA  research  aircraft  was  designed  to  allow  control  systems 
engineers  the  resources  adequate  for  modeling  flight  control  systems  under  these 
conditions. 

Failure-tolerant  control  systems  can  be  characterized  as  robust,  reconfigurable,  or 
some  combination  of  the  two.  A  system  that  retains  satisfactory  performance  in  the 
presence  of  input  parameter  variations  or  component  failures  is  called  robust.  If  the 
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control  system's  structure  or  parameters  alter  in  response  to  system  failure,  it  is 
reconfigurable.  In  the  case  of  reconfigurable  control  systems,  the  control  system  detects, 
identifies,  and  isolates  failures,  and  then  it  modifies  control  laws  as  required  to  maintain 
acceptable  stability,  trim,  and  performance. 

Early  VISTA  research  focused  on  methods  of  designing  robust  flight  control  systems 
for  the  F-22  that  compensate  for  potentially  catastrophic  failures  of  aircraft  systems — 
especially  during  combat.  Robust  flight  control  methods  are  usually  based  on  redundant 
systems,  “expert  systems”,  and  probabilistic  evaluations  of  control  responses.  The  F-22 
program  plans  to  eventually  incorporate  “expert  systems”  into  the  aircraft  control  system, 
but  initially  it  will  rely  on  redundancy  and  probabilistic  evaluations.  Probabilistic  data 
also  helps  predict  the  effects  of  system  failures  on  man-in-the-loop  stability  and 
performance  through  assessment  of  commonly  accepted  design  criteria  outlined  in  Mil- 
Std  1797A  Flying  Qualities  for  Piloted  Aircraft. 

VISTA  simulations  of  control  systems  designs  determined  the  likelihood  of  system 
responses  and  whether  the  proposed  F-22  control  systems  responses  fell  within  Mil-Std 
1797A  guidelines.  This  research  was  a  major  step  toward  computerized  design  of  digital 
flight  control  systems.  The  computerized  design  can  eventually  be  incorporated  into  a 
larger  digitized  aircraft  design  processes  such  as  the  one  used  to  design  the  Boeing  777 
described  in  the  next  chapter. 

The  principal  functions  of  the  F-22  failure  tolerant  flight  control  system  are  failure 
detection  through  built-in-test  alarms,  off-nominal  operations,  component  failure 

Notes 

36  Robert  F.  Stengel,  "Failure-Tolerant  Flight  Control  Systems"  FAA  Grant  No.  95-G- 
022,  24  July  1997 ,  n.p.;  on-line,  Internet,  available  from  http://www. 
prince  ton  .edu/~  stengel/FTFCS .  html . 
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identification,  and  control- system  reconfiguration.  Detection  and  identification  are 
combined  in  the  built-in  test  functions.  Although  in-line  monitors  provide  direct  and 
rapid  response  to  specific  failures,  it  is  impossible  to  provide  full  coverage  of  all  failures 
by  instrumentation,  which  is  also  subject  to  failure.  However,  a  practical  failure 
detection,  identification,  and  reconfiguration  (FDIR)  solution  could  be  found  by  using 
M&S  technologies.  The  F-22  flight  control  system  compares  expected  response  from  the 
VISTA  data  to  the  actual  F-22  response,  inferring  component  failures  from  the 
differences  and  changing  either  the  configuration  of  the  aircraft  or  the  parameters  of  the 
control  system. 

The  F-22  failure  detection  algorithms  are  exemplified  by  a  generalized  likelihood 
ratio  test,  which  uses  a  Kalman  filter-like  recursive  equation  to  sense  discrepancies  in  the 
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system  response.  This  test  compares  the  probability  of  the  estimator's  actual 
measurement  residual  with  its  expected  value,  detecting  a  jump  or  discontinuity  that 
indicates  a  possible  failure.  Because  variations  in  the  measurement  residuals  were 
extensively  tested  in  the  VISTA,  these  indicators  are  very  accurate  and  sensitive  to  off 
nominal  performance.  The  F-22  FDIR  system  is  very  sensitive  to  off-nominal 
performance  and  failures  are  easy  to  detect.  However,  the  VISTA  tests  did  not  produce 
precise  indications  of  the  specific  failed  components.  Modeling  errors  still  occasionally 

iq 

hamper  detection. 

The  crash  of  an  F-22  during  flight  test  due  to  a  control  system  failure  also  reminds  us 
that  further  improvement  in  the  design  of  the  F-22’s  digital  flight  control  systems  is 

Notes 

Charles  W.  Brown,  Lockheed  Corp.,  interviewed  by  author,  7  March  1998. 

38  4  1  6  Flight  Test  Squadron,  “F-22  Software  Failure,  Detection,  Identification,  and 
Reconfiguration  (FDIR)  Test  Plan,”  Edwards  AFB,  CA.  Jun  1996.  38. 
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warranted.  In  this  case,  the  cause  of  the  failure  was  determined  to  be  missing  computer 
code  in  flight  control  system.  Because  the  software  is  modified  occasionally  throughout 
the  lifetime  of  the  aircraft,  there  is  a  continuing  need  to  assess  and  prevent  potentially 
catastrophic  accidents  as  the  result  of  control  system  failures.  The  VISTA  allowed 
engineers  to  flight-test  changes  to  the  control  systems  software  used  in  the  F-22.  In-flight 
simulation  permitted  the  Air  Force  to  assess  the  F-22  fly-by-wire  control  systems 
conveniently  and  safely. 

The  F-22  uses  models  developed  during  the  VISTA  flight-tests  to  reconfigure  flight 
controls  in  order  to  retain  nominal  stability,  performance,  and  trim  characteristics. 
However,  there  is  a  tradeoff  between  speed  of  reconfiguration,  computer  storage 
requirements,  and  flexibility  of  reaction.  Models  and  parameters  for  all  conceivable  failed 
states  could  be  generated  off-line  and  stored  for  eventual  use.  However,  this  approach 
would  require  an  enormous  amount  of  memory.  Conversely,  adaptation  requires  minimal 
data  storage  and  theoretically  could  adjust  to  unanticipated  failures,  but  design  algorithms 
must  be  executed  and  their  results  accepted  soon  enough  to  provide  sufficient  failure 
tolerance.40  Because  the  aircraft  is  open-loop-unstable,  higher  control  activity  combined 
with  the  control  saturation  limits  reduce  the  solution  space  within  which  closed-loop 
stability  can  be  assured. 


Notes 

Fred  Carmody,  Lockheed  Corp.,  interviewed  by  the  author,  1  April  98. 

40  Robert  F.  Stengel,  "Failure-Tolerant  Flight  Control  Systems"  FAA  Grant  No.  95-G- 
022,  24  July  1997 ,  n.p.;  on-line,  Internet,  available  from  http://www. 
princeton.edu/~stengel/FTFCS.html. 
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Evaluating  M&S  for  Flight-Testing  Closed-Loop  Control  Systems 

M&S  provide  an  important  method  for  evaluating  the  flying  qualities  of  aircraft 
without  putting  an  expensive  prototype  aircraft  at  risk.  The  F-22  costs  the  Air  Force  at 
least  $20,000  per  hour  to  flight-test.  The  VISTA  costs  $6,000  to  $8,000  per  flight  hour, 
depending  on  the  specific  software  and  instrumentation  requirements.41  The  VISTA  also 
has  the  ability  to  change  programs  in  the  VSS  system  in-flight.  Therefore,  the  flight-test 
team  has  the  ability  to  fly  several  simulations  using  one  airframe  and  one  sortie. 

The  Air  Force  has  already  lost  one  F-22  to  flight-control  problems.  The  crash 
resulted  in  the  loss  of  an  $83  million  asset  and  put  the  program  behind  schedule. 
Fortunately,  there  was  no  loss  of  life  in  this  accident.  Nevertheless,  this  mishap  is  an 
important  reminder  that  flight-te sting  flight  control  systems  can  be  very  dangerous.  In 
this  case,  the  cost-benefit  analysis  is  straightforward.  It  is  cheaper  and  safer  to  flight-test 
in  the  VISTA  than  in  the  F-22. 

Digital  flight-control  systems  can  be  especially  dangerous,  because  software  faults 
are  so  difficult  to  detect  and  can  open  a  feedback  control  loop  on  an  open-loop-unstable 
aircraft.  As  long  as  the  in-flight  simulator  provides  valid  engineering  data,  simulation  is 
preferable  to  flight-testing  with  an  F-22.  The  data  provided  by  the  VISTA  was  crucial  to 
designing  robust  flight  controls  for  the  F-22.  The  role  software  plays  in  determining  the 
airworthiness  of  a  control  system  design  cannot  be  overemphasized.  Small  changes  to 
software  can  drive  large  changes  to  the  flight  control  actuators  and  adversely  affect  the 
aircraft’s  flying  qualities. 

Note 

41  These  figures  are  the  best  available  estimates  as  of  1  May  1998  from  the  AFFTC 
resources  branch. 
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In-flight  simulation  is  an  important  capability.  If  the  analysis  of  aircraft  dynamics  is 
poor  on  the  ground,  the  actual  aircraft  dynamic  response  is  likely  to  be  even  worse  in 
flight.  A  corollary  to  this  lies  at  the  core  of  the  VISTA’s  mission.  Even  if  the  computed 
aircraft  dynamics  and  flying  qualities  are  very  good  on  the  ground — such  as  the  F-22 — 
the  handling  qualities  are  not  necessarily  good  in  the  air. 

The  only  way  to  compare  computed  dynamic  responses  with  actual  aircraft  response 
is  by  flight-test.  In-flight  simulation  is  the  only  way  to  flight-test  digital  control  systems 
safely.  VISTA's  probabilistic  models  of  the  F-22  flight  parameters  and  control  responses 
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matched  the  F-22  data  almost  exactly — within  the  flight  envelope  of  the  VISTA. 
Therefore,  the  engineering  validity  of  the  VISTA  data  was  excellent,  but  the  VISTA 
could  not  provide  a  comprehensive  model  data  because  of  its  limited  flight  envelope. 

Although  the  engineering  data  provided  by  the  VISTA  was  excellent,  the  VISTA  did 
not  have  as  large  a  performance  envelope  as  the  F-22.  Therefore,  the  VISTA  model  was 
not  comprehensive.  The  VISTA  could  only  model  a  subset  of  the  actual  F-22  responses. 
At  high  Mach  numbers,  the  VISTA  could  not  provide  modeling  data  because  the  aircraft 
is  limited  to  1.6  mach  although  the  F-22  can  attain  2.0  mach  number.  Fikewise  at  high 
altitudes,  the  VISTA  could  not  simulate  certain  super-cruise  conditions  because  of  engine 
limitation  on  the  VISTA's  engine  that  prevented  the  initiation  of  afterburner  above 
approximately  38,000  feet.43 

Fortunately,  the  limitations  of  the  VISTA  were  known  by  the  flight-test  team. 
Therefore,  the  VISTA  model  of  the  F-22  control  system  was  simply  a  subset  of  the  actual 

Note 

42 

"  Wilson  and  Hutchinson,  81. 

43  Karl  T.  Hutchinson,  ed.,  VISTA/F-16  Variable  Stability  System  Pilots’  Operating 
Manual,  CAF-7693-9  (Buffalo,  NY.  January  1992),  5-3. 
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F-22  control  responses.  Although  the  model  was  not  comprehensive,  no  errors  of 
omission  were  made  interpreting  the  VISTA  data,  and  the  F-22  did  not  have  unexpected 
control  responses. 

Because  the  Air  Force  already  has  extensive  experience  with  in-flight  simulation, 
many  of  the  lessons  learned  during  the  development  of  the  F-22  flight  control  system  are 
lessons  relearned  and  reinforced  from  the  earlier  experience  with  in-flight  simulator 
programs.  The  VISTA  provides  a  cost-effective  and  valid  model,  but  not  necessarily  a 
comprehensive  one.  In-flight  simulation  is  already  an  integral  part  of  designing  the  flight 
control  system  for  a  digital  fly-by-wire  aircraft.  Very  soon,  in-flight  simulation  will 
become  part  of  an  overall  digitized  aircraft  design  process,  such  as  the  Boeing  777  case 
described  in  the  next  chapter.  In-flight  simulation  is  the  only  way  to  safely  flight-test 
digital  flight-control  systems.  The  Air  Force  should  continue  to  promote  and  use  this 
important  M&S  technology. 
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Chapter  4 


Case  Study  Three-  Modeling  Complex  Systems 
by  Designing  the  Boeing  111  in  Cyberspace 


Modeling  and  simulation  will  improve  readiness  and  reduce  costs  for  the 
nation  because  it  will  allow  us  to  demonstrate  the  flexibility, 
responsiveness,  and  utility  of  airpower... 

General  Ronald  R.  Fogelman 
USAF  Chief  of  Staff 


Introduction 

The  previous  case  studies  demonstrated  how  the  capabilities  and  limitations  of  M&S 
vary  as  models  increase  in  scope  and  complexity  from  the  physical  component  level  to 
closed-loop  control  systems.  The  B-777  case  increases  the  scope  and  complexity  of  the 
modeling  effort  by  modeling  a  complex  system  with  human  interactions.  Recent 
advances  in  M&S  technologies  have  made  significant  changes  in  the  way  the  Boeing 
Aircraft  Company  designs,  builds,  and  tests  airplanes.  The  Boeing  777  (B-777)  was  the 
first  aircraft  created  using  M&S  methodologies  as  the  foundation  of  their  design  and 
engineering  processes.  The  changes  wrought  by  M&S  were  very  dramatic  and 
encompassed  many  areas,  including  technical,  organizational  and  administrative  changes. 
Consequently,  Boeing  has  touted  the  B-777  as  more  than  just  a  product,  but  rather  a  new 
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process  for  the  company.44  Although  the  technical  innovations  were  impressive,  what 
made  the  B-777  project  unique  was  the  way  that  Boeing  integrated  state  of  the  art  M&S 
technologies  throughout  the  design,  production,  and  testing  of  the  aircraft. 

Historical  Background  of  the  B-777  Program 

In  the  early  1990s,  Boeing  was  beginning  the  concept  development  for  its  next 
generation  commercial  airliner.  The  company  was  losing  its  market  share  for  commercial 
airliners  to  the  Airbus  consortium  in  Europe  and  US  air  carriers  were  finding  the 
advanced  technology  incorporated  into  the  Airbus  320  very  attractive  for  the  US 
domestic  market.  Because  the  Airbus  consortium  enjoys  financial  backing  from  European 
governments,  Boeing  was  under  intense  pressure  to  reduce  its  operating  costs  in  order  to 
be  financially  competitive.  Nevertheless,  both  the  aircraft  and  the  production  methods 
needed  to  be  state  of  the  art  in  order  for  Boeing  to  be  technologically  competitive.  M&S 
was  the  key  technology  that  enabled  the  B-777  program  to  meet  both  imperatives.  M&S 
reduced  costs  and  eliminated  the  costly  mockup  and  prototyping  stages  of  the  aircraft 
design  process. 

Designing  and  building  a  new  commercial  jet  airliner  is  a  long  process,  sometimes 
lasting  five  to  ten  years.  The  process  is  relatively  infrequent  and  expensive  as  well.  The 
Boeing  Company  could  only  expect  to  go  through  the  process  of  designing  and  building  a 
new  airliner  once  or  twice  per  decade.  Therefore,  senior  Boeing  management  believed 
that  it  was  critical  to  document  the  new  M&S  design  and  production  innovations  for 

Note 

44  Shad  H  Shokralla,  "21st  Century  Jet,  The  Boeing  777,"  National  Engineering 
Education  Delivery  System  26  January  1997 ,  n.p.;  on-line,  Internet,  1  June  1998, 
available  from  http://wwwl.needs.org/develop/b777. 
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future  projects.45  This  was  especially  true  for  the  B-777  program  because  of  the  many 
technological  “firsts.”  Fortunately,  M&S  technologies  enhanced  the  documentation  of  the 
design  process  due  to  the  relatively  inexpensive  storage  and  archival  of  computer  data. 
Computer  Aided  Design  (CAD)  also  enhanced  the  design  process  because  it  was  easier 
and  faster  to  learn  than  traditional  engineering  drawing  methods.  Over  the  course  of  the 
B-777  program,  the  computer  models  became  a  part  of  Boeing’s  “corporate  memory.” 
The  models  will  undoubtedly  continue  to  play  an  important  part  in  future  aircraft  design 
programs. 

However,  when  modeling  a  complex  system  and  trying  to  account  for  human 
interaction  with  that  system,  M&S  methods  currently  have  significant  limitations.  Boeing 
pushed  the  state  of  the  art  in  M&S  for  the  B-777  program  and  ran  up  against  some  of 
these  limitations.  The  B-777  program  made  significant  engineering  oversights  and  errors 
of  omission.  The  company  also  had  managerial  difficulties  because  of  the  self-imposed 
dependence  on  computer  technology.  Ultimately,  M&S  could  not  do  it  all  on  a  project  as 
complex  as  the  B-777.  However,  M&S  was  an  extraordinarily  valuable  tool,  and  the 
decision  to  use  M&S  methods  turned  out  to  be  a  successful  solution  for  the  company 
despite  the  difficulties  and  occasional  oversights. 

The  Modeled  SystemBoeing  777  Aircraft  Description 

In  order  to  understand  the  process  of  using  M&S  on  a  major  program,  it  is  important 
to  know  the  product.  In  this  case,  the  product  is  a  new  jet  airliner  with  numerous 
technical  innovations.  The  B-777  is  the  biggest  twin-engine  aircraft  ever  to  fly.  The  B- 

Note 

45  John  E.  Cashman,  "Progress  to  the  Plan,"  Proceeding  of  the  39th  Annual  Symposium  of 
the  Society  of  Experimented  Test  Pilots,  1  September  1995,  7. 
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777  wing  span  is  almost  200  feet — equal  to  the  Boeing  747’s  wingspan.  The  B-777’s 
nacelle  is  twelve  feet  in  diameter,  which  is  approximately  the  diameter  of  a  Boeing  757 
fuselage.  The  B -777-200  is  also  the  first  twin-engine  aircraft  to  have  a  range  of  4,000  to 
8,500  nautical  miles  carrying  between  300  and  400  passengers.  The  aircraft  was 
classified  as  a  very  heavy  or  jumbo-jet,  which  places  it  in  the  500,000  to  660,000  lb. 
weight  class.  The  initial  model  of  the  B-777  flies  375  passengers  4,630  nautical  miles.  An 
increased  gross  weight  model  will  carry  305  passengers  7,230  nautical  miles.  This  is  the 
preferred  configuration  for  long,  intercontinental  routes.  Future  members  of  the  B-777 
family  will  carry  as  many  as  550  passengers  and  fly  farther  than  the  current  747-400 
model.  The  aircraft  entered  commercial  service  in  June  1995,  and  it  has  been  certified  for 
extended-range  twin-engine  operations  (ETOPS).  Additional  technical  information  about 
the  aircraft  is  located  at  Appendix  C.46 

During  the  design  definition  stage  of  the  B-777,  many  new  technologies  were 
investigated  and  discussed  with  the  airlines.  According  to  Boeing’s  Vice-President  for  the 
B-777  project,  Phil  Condit:47 

“  The  decider  in  this  area  was  to  select  only  the  technologies  that 
“bought”  their  way  onto  the  airplane.  That  is  the  technologies  that  would 
provide  an  economic  or  safety  return  to  the  airline,  as  well  as  ones  they 
were  currently  ready  to  incorporate  into  their  internal  systems.  We  are  also 
attempting  to  build  a  system  that  could  be  upgraded  for  other  known  and 
future  digital  technologies  such  as  an  electronic  library  system  for  on¬ 
board  technical  information,  differential  GPS,  and  HUD  to  cite  a  few.” 

Among  the  new  technologies  to  make  the  “cut”  onto  the  airplane  design  were  a 
digital  avionics  suite,  high-speed  data  buses,  centralized  Airplane  Information 

Note 

46  ibid.  7. 

47  Shad  H  Shokralla,  "21st  Century  Jet,  The  Boeing  777,"  National  Engineering  Delivery 
System  26  January  1997,  n.p.;  on-line,  Internet,  1  June  1998,  available  from 
http  ://w  w  w  1 .  needs .  org/de  velop/b7 7 7 . 
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Management  System  (AIMS),  LCD  flat  panel  displays,  and  a  new  generation  of  high 
bypass-ratio/high-thrust  engines. 

The  B-777  also  used  an  advanced  fly-by-wire  flight  control  system.  Essentially,  fly¬ 
by-wire  technology  is  a  system  of  transmitting  flight  control  commands  from  the  cockpit 
via  data  cables  to  actuators  on  the  control  surfaces.  Primary  flight  control  computers 
convert  the  pilot's  movement  of  the  control  column  and  rudder  pedals  into  movements  of 
the  aerodynamic  control  surfaces  via  input-response  models  stored  in  the  computer 
memory.  This  means  that  there  are  no  direct  links — either  mechanical  or  hydraulic — 
between  the  pilots  and  the  control  surfaces.  The  computers  also  determine  the  force  and 
displacement  necessary  at  the  control  column  and  rudder  pedals  to  provide  an  artificial 
feel  system  for  the  flight  crew.  Fly-by-wire  systems  eliminate  weight  and  reduce  control 
system  complexity  by  removing  a  majority  of  the  cables,  linkages,  and  hydraulic  tubing. 
Furthermore,  since  computer  models  drive  the  control  actuators  from  pilot  inputs,  the 
aircraft’s  flying  qualities  can  be  fine-tuned  with  software  changes  for  safer  and  smoother 
aircraft  control.49 

Fly-by-wire  is  not  a  new  technology.  It  has  been  used  in  military  aircraft  for  years. 
The  F-16  was  the  first  production  aircraft  to  use  fly-by- wire  technology  with  an  analog 
system  introduced  in  the  late  1970s.  The  Airbus  consortium  has  also  used  a  hybrid 
analog/digital  fly-by- wire  system  in  the  Airbus  320  since  1985.  The  Airbus  320  uses  a 
two-axis  digital  system  in  which  the  yaw  controls  were  analog.  However,  the  B-777  was 
the  first  commercial  aircraft  to  incorporate  a  completely  digital  three-axis  fly-by-wire 

Note 

48  Cashman,  10. 

49  Jan  Roskam,  Airplane  Flight  Dynamics  and  Automatic  Flight  Controls  (Fawrence  KS: 
Design,  analysis,  and  Research  Corporation.,  1994),  183-186. 
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system.  The  Boeing  system  was  also  the  first  production  fly-by-wire  control  system  that 
blended  inputs  to  the  flight  control  software.50  This  ability  to  smoothly  blend  control 
inputs  is  similar  to  the  VISTA  in-flight  simulator  discussed  in  the  previous  case.  The  B- 
777  fly-by-wire  system  typically  used  a  blend  of  normal  aircraft  acceleration,  airspeed, 
and  pitch  rate  to  model  and  compute  aircraft  response  to  pilot  inputs.  This  technique  of 
blending  inputs  to  the  flight  control  laws  as  part  of  a  control-response  model  in  the  flight 
control  computers  resulted  in  significantly  improved  handling  qualities  over  previous  fly- 
by- wire  systems. 


Innovative  Design  Techniques 

In  the  past,  Boeing  had  always  made  thousands  of  engineering  drawings  and  then 
built  a  full-scale  non-flying  mockup  of  the  aircraft  to  check  fit  and  interference  problems. 
In  the  case  of  the  B-777,  the  cost  of  the  mockup  alone  was  estimated  to  have  been  at  least 
$22.5  million.51  Without  M&S  technologies  and  the  computer  design  network,  all  the 
various  aircraft  systems  must  be  designed  independently. 

Therefore,  effective  systems  integration  required  a  physical  mockup  before  the 
aircraft  could  enter  production.  For  example,  the  physical  mockup  made  sure  that  a  bolt 
did  not  occupy  the  same  physical  space  as  a  hydraulic  line,  or  that  an  electrical  conduit 
did  not  run  across  the  middle  of  a  ventilation  duct.  The  mockup  was  also  used  to  check 
accessibility  of  all  the  parts  for  maintenance  work. 


Note 

50  Cashman,  6. 

51  Shad  H  Shokralla,  "21st  Century  Jet,  The  Boeing  777,"  National  Engineering  Delivery 
System  26  January  1997,  n.p.;  on-line,  Internet,  1  June  1998,  available  from 
http  ://w  w  w  1 .  needs .  org/de  velop/b7 7 7 . 
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However,  the  B-777  program  avoided  the  necessity  of  mockups  by  using  computer 
aided  design  (CAD)  methodologies.  CAD  replaced  the  tedious  hand  drawn  layouts  and 
full-size  mockups  of  earlier  aircraft  designs.  Boeing  used  an  innovative  combination  of 
networked  computer  systems,  M&S  technology,  and  Design/Build  Teams  (DBTs)  to 
coordinate  design  requirements  and  engineering  priorities. 

In  order  to  accomplish  this  coordination  at  the  component  level,  the  B-777  division 
used  a  CAD  software  suite  based  on  CATIA  (Computer-Aided,  Three-dimensional 
Interactive  Application)  and  a  finite  element  analysis  program  known  as  ELFINI.  Both 
programs  were  developed  by  Dassault  Systems  of  France  and  licensed  in  the  United 
States  through  IBM.  ~  CATIA  was  used  for  the  all  the  detailed  computer  design  work,  as 
well  as  the  three-dimensional  solid  modeling.  At  the  aircraft  systems  level,  Boeing 
designers  also  used  EPIC  (Electronic  Pre-assembly  Integration  on  CATIA)  as  well  as 
other  digital  pre-assembly  applications  developed  by  Boeing  specifically  for  the  B-777 
project.53 

In  order  to  model  the  B-777  as  a  complex  system,  Boeing  connected  2,200  CATIA 
workstations  into  an  eight-mainframe  computing  cluster.  This  mainframe  cluster  was  also 
linked  to  mainframes  and  CATIA  workstations  throughout  the  world.  This  network 
required  the  largest  data  storage  capacity  of  any  computing  facility  in  the  world.  The  B- 


Note 

52  Michael  Mecham,  "Aerospace  Chases  the  Software  Boom,"  Aviation  Week  and  Space 
Technology  Vol.  147  no.  14  (Oct.  6,  1997):  47. 

Shad  H  Shokralla,  "21st  Century  Jet,  The  Boeing  777,"  National  Engineering  Delivery 
System  26  January  1997,  n.p.;  on-line,  Internet,  1  June  1998,  available  from 
http  ://w  w  w  1 .  needs .  org/de  velop/b7 7 7 . 
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777  division  required  over  three  tera-bytes  (3x10  ")  of  data  to  store  all  the  aircraft  design 


information.54 

M&S  technologies  are  more  than  three-dimensional  design  tools,  when  dealing  with 
complex  systems.  CATIA  was  used  as  a  component  level  modeling  method  as  well  as  a 
digital  pre-assembly  tool.  The  DBTs  used  this  data  in  conjunction  with  a  computer 
network  to  produce  a  "paperless"  design  that  also  allowed  engineers  to  simulate  the 
complete  assembly  of  the  B-777.  By  using  the  three-dimensional  solid  images  generated 
on  a  computer,  the  B-777  airplane  could  be  pre-assembled  in  cyberspace  to  position  parts 
properly  and  to  ensure  a  good  fit.  Additional  software  tools  allowed  for  “fly-through” 
analysis  of  various  hardware  configurations.  Human  factors  and  maintenance  access 
questions  were  answered  by  maneuvering  a  digital  "virtual  mechanic"  in  three- 
dimensional  space  (see  Figure  13).  Many  complex  systems  issues  including  some  human 
factors  and  maintenance  accessibility  studies  could  be  done  from  individual 
workstations.55  With  a  three-dimensional  database  that  everyone  could  use 
simultaneously,  design  interference  problems  were  greatly  reduced  before  full-scale 
production. 


Note 

54  Cashman,  12. 

55  Shad  H  Shokralla,  "21st  Century  Jet,  The  Boeing  777,"  National  Engineering  Delivery 
System  26  January  1997,  n.p.;  on-line,  Internet,  1  June  1998,  available  from 
http://wwwl.needs.org/develop/b777. 
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Figure  13:  A  “virtual”  mechanic  inspects  the  landing  gear 
(courtesy  of  Boeing  Corp.) 

Another  benefit  of  M&S  technology  was  that  the  CATIA  system  also  allowed  flight- 
test  instrumentation  to  be  designed  into  a  standard  airplane  during  normal  factory 
production.  This  resulted  in  a  more  efficient  design  process,  and  the  elimination  of  even 
more  specialized  mockups  for  test  instrumentation.  According  to  Boeing’s  Chief  of  Flight 
Test  Operations,  Mr.  John  Cashman  “CAD  design  methods  meant  less  rework  of  normal 
aircraft  systems  when  test  instrumentation  was  installed.”56 

Modeling  &  Simulation  Methods 

Systems  Level  Modeling  and  Knowledge  Based  Engineering 

Knowledge  Based  Engineering  (KBE)  on  the  B-777  program  also  took  M&S 
technologies  a  step  beyond  component  level  models  and  into  systems  level  modeling 

Note 

56  Christoph  W.  Klomp,  Boeing  Corp.,  interviewed  by  author,  15  March  1998. 
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methods.  KBE  not  only  captured  the  design  information  but  also  the  rules  that  were  used 
to  create  the  design.  KBE  is  also  known  as  an  "expert"  system.  An  “expert”  system  is  a 
computer  programmed  to  mimic  the  procedures  and  decisions  that  an  experts  would 
make.  For  example,  KBE  uses  data  about  material  expense,  strength,  and  weight  to 
determine  if  it  meets  the  requirements  for  a  specific  part.  An  “expert”  system  can  make 
design  decisions  within  a  engineering  design  framework  and  is  useful  in  a  large 
engineering  project  because  “expert”  systems  allow  the  computer  model  to  keep  track  of 
multitudinous  design  rules. 

Boeing’s  KBE  system  utilized  CAD  software  from  Concentra  Inc.,  in  conjunction 
with  computer  code  developed  in-house  by  Boeing.57  The  KBE  software  was  written  in 
LISP — a  common  object-oriented  computer  language  favored  for  expert-system 
applications.  Expert  systems  using  standardized  design  rules  generally  lead  to  more  part 
commonality  and  reduce  manufacturing  tooling  costs.  For  example,  stringers  in  a  wing 
are  similar  structural  elements  to  ribs  in  a  fuselage.  However,  they  were  often  designed 
using  different  criteria  based  on  the  experience  gained  by  many  designers  on  the  different 
fuselage  and  wing  design  teams.  The  KBE  methods  made  sure  that  the  lessons  learned  on 
one  design  project  were  not  lost  on  another.  KBE  also  helped  prevent  the  duplication  of 
effort  during  engineering  design  and  analysis  of  similar  parts. 

However,  KBE  is  not  a  completely  mature  modeling  methodology  and  the  rules  used 
by  “expert”  systems  sometimes  work  at  cross-purposes.  For  example,  design  safety 
margins  vary  widely  on  structural  parts.  Fuselage  stringers  along  the  top  of  the  aircraft 
must  be  as  much  as  50  percent  stronger  than  the  stringers  on  the  bottom  of  the  fuselage. 

Note 
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However,  the  engineering  drawings  are  the  same  for  both  parts.  During  the 
manufacturing  process  extra  metal  is  carefully  machined  away  from  the  lower  stringers  to 
save  weight.  An  “expert”  system  must  be  able  to  adjust  for  the  different  requirements  for 
each  part,  and  then  be  able  to  label  and  track  the  part  through  the  production  phase.  KBE 
worked  well  for  the  wing  parts  but  needed  a  much  more  comprehensive  database  for 
fuselage  components.  In  the  case  of  the  B-777,  some  fuselage  parts  were  over-built 
because  of  the  inadequate  database.  Boeing  knew  the  KBE  model  was  still  incomplete  at 

r  o 

the  systems  level,  but  decided  to  continue  using  it  in  order  to  save  production  time. 
However,  these  errors  of  omission  at  the  systems  level  resulted  in  a  significant  weight 
penalty  on  the  final  aircraft  design. 

On  the  other  hand,  one  of  the  major  benefits  of  KBE  was  the  ability  to  run  computer 
simulations  to  optimize  design  based  on  KBE  logic  and  conditional  rules.  This  capability 
allowed  designers  to  conduct  additional  trade-off  studies  and  look  at  more  options,  which 
resulted  in  a  better  design  with  reduced  material  and  complexity.  Standardized 
manufacturing  rules  in  the  system  resulted  in  designs  that  were  easier  and  cheaper  to 
manufacture.  The  KBE  system  also  automatically  generated  parts  lists  to  facilitate 
production. 

KBE  was  not  only  used  in  engineering  and  structural  design  but  also  in  customizing 
each  cabin  configuration.  All  the  rules  for  the  dimensions  and  spacing  of  seats,  aisles, 
lavatories  and  galleys  were  entered  into  the  system  which  allowed  each  airline  to  quickly 
determine 


Note 
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the  maximum  number  of  seats  and  locations  of  all  the  components  for  their  chosen 
configuration  (see  Figure  14).  This  process  helped  reduce  the  time  required  for 
configuration  changes  to  approximately  three  days  as  opposed  to  three  weeks  to  a  month 


without  M&S.59  As  far  as  the  passengers  were  concerned,  this  meant  is  that  individual 
seat  selection  was  now  an  important  issue.  The  same  class  seats  on  the  same  flight  could 
vary  by  one  to  two  inches  in  both  legroom  and  seat  width  because  the  computer  was 


Figure  14:  Computer  Ponders 

making  the  most  efficient  use  of  space — not  necessarily  the  most  equitable  one. 

The  FAA  requirements  for  the  final  stress  analysis  were  also  input  to  the  KBE.  This 
meant  that  FAA  requirements  were  design  rules  from  the  beginning.  Therefore,  critical 
engineering  deficiencies  could  be  found  and  corrected  much  earlier  than  the  prototype  or 


59 


test  phase.  For  example,  the  maximum  landing  gear  side  loads  during  crosswind  landings 
was  an  engineering  specification.  KBE  used  “expert”  system  to  design  the  landing  gear 
system,  and  the  model  of  the  complete  aircraft  prevented  the  engineering  specification 
from  being  violated  in  the  final  design.60  This  complete  aircraft  model  was  developed 
using  the  technique  of  concurrent  engineering. 

Complex  Systems  Modeling  and  Concurrent  Engineering 

At  the  next  level  of  modeling  complexity,  the  complex  systems  model,  Boeing  used 
concurrent  engineering  methods.  Concurrent  engineering  is  the  concept  of  “working 
together  in  cyberspace,”  and  was  an  integral  part  of  the  B-777  design  methodology.61  The 
basic  idea  behind  concurrent  engineering  is  to  take  advantage  of  KBE  and  networked 
M&S  capabilities.  Design  and  manufacturing  teams  worked  concurrently  on  the  design  to 
decrease  later  change  orders  and  to  increase  efficiency  in  production. 

Concurrent  engineering  was  not  a  new  concept  or  revolutionary  idea,  but  for  Boeing 
it  was  a  completely  new  way  of  doing  business.  According  to  Boeing’s  Vice-President 
for  the  B-777  project,  Phil  Condit  “Boeing  is  a  conservative  company  that  dislikes 
change”  and  both  the  manufacturing  and  design  teams  were  comfortable  with  traditional 
engineering  and  manufacturing  methods.  The  old  stovepipe  design  teams  and  "throw  it 
over  the  wall  method"  technique  seemed  to  work  well  enough  in  the  past.  Stovepipe 
design  methods  also  gave  both  design  and  manufacturing  engineers  a  ready  scapegoat 
for  errors.  However,  this  was  not  an  efficient  design  or  production  process.  Boeing  Vice- 

Notes 

5y  Shad  H  Shokralla,  "21st  Century  Jet,  The  Boeing  777,"  National  Engineering  Delivery 
System  26  January  1997 ,  n.p.;  on-line,  Internet,  1  June  1998,  available  from 
http://wwwl.needs.org/develop/b777. 

60  Richard  W.  Schmidtt,  Boeing  Corp.,  interviewed  by  author,  17  April  1998. 

61  Christoph  W.  Klomp,  Boeing  Corp.,  interviewed  by  author,  15  March  1998. 
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President  Phil  Condit  pointed  out  that,  on  average,  every  drawing  in  the  old  Boeing 
system  was  changed  four  and  one  half  times  before  final  release.  According  to  Condit, 
"Boeing  designed  four  and  a  half  planes  for  every  plane  we  built." 

Concurrent  engineering  saved  a  great  deal  of  time  by  preventing  error  and  rework, 
but  it  required  heavier  emphasis  on  planning  at  an  earlier  stage.  This  saved  many  design 
iterations  because  the  team’s  product  did  not  have  to  go  from  the  designer  to  planner  to 
manufacturer  to  assembly  and  back  again  as  everyone  uncovered  imperfections  to  the 
original  design  in  turn.  Because  all  the  parties  were  involved  from  the  beginning,  the 
design  did  not  leave  the  team  without  being  reviewed  by  everyone  involved. 

Prior  aircraft  development  programs  suffered  from  changing  requirements  and  high 
redesign  costs.  Design  requirements  were  constantly  changing  especially  as  full-scale 
mockups  were  built  to  check  form,  fit,  and  function  of  the  individual  aircraft  components. 
Redesign  meant  costly  delays  not  just  in  the  months  leading  to  the  first  flight,  but 
throughout  the  flight-test  and  certification  process. 

Fortunately,  M&S  technologies  allowed  Boeing  to  overcome  many  of  the  difficulties 
and  delays  inherent  in  traditional  design  practices.  The  B-777  was  the  first  aircraft  ever  to 
be  100  percent  digitally  designed  using  three-dimensional  solids  modeling  technology. 
Because  the  individual  components  were  stored  as  three-dimensional  computer  models, 
the  form,  fit,  and  function  of  the  individual  components  could  be  checked  with  M&S 
tools  at  a  computer  workstation.  Throughout  the  design  process,  the  airplane  was  "pre¬ 
assembled"  on  the  computer  workstations,  eliminating  the  need  for  costly  full-scale 
mockups,  saving  at  least  $22.5  million  in  program  costs. 
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Because  the  CATIA  workstations  were  networked,  it  was  also  easy  to  coordinate 
design  changes  between  the  teams.  The  same  M&S  techniques  used  during  the  design 
phase  also  allowed  a  “virtual”  B-777  airplane  to  be  digitally  “pre-assembled”  in 
cyberspace.  M&S  allowed  the  teams  to  study  the  effects  of  design  changes  on  the  aircraft. 
Design  changes  could  be  coordinated  digitally  before  building  expensive  prototypes  and 
then  redesigning  for  unforeseen  changes.  Concurrent  engineering  permitted  a  more 
mature  and  stable  design  to  be  reached  sooner.  Therefore,  Boeing  decided  to  set  a  very 
aggressive  manufacturing  and  flight-test  schedule  for  the  B-777. 

The  CATIA  network  performed  so  well  at  the  engineering  component  level  of  M&S 
that  only  a  nose  mockup  (to  check  critical  wiring)  was  built  before  assembly  of  the  first 
flight  vehicle.  The  center  of  the  nose  mockup  was  only  0.03  mm  out  of  alignment  with 
the  middle  of  the  fuselage  as  measured  on  the  waterline  of  the  aircraft.  This  is 
approximately  an  order  of  magnitude  improvement  over  traditional  engineering  and 
prototyping  methods  without  CAD  design.  During  initial  production,  the  engineering 
results  of  this  CAD  design  work  were  also  quite  impressive.  For  example,  the  port  wing 
tip  of  the  first  B-777  was  out  of  position  by  approximately  one-thousandths  of  an  inch 
when  attached  to  the  fuselage,  and  the  starboard  wing  was  located  within  the  accuracy  of 
the  laser  theodolites  used  for  alignment.  In  addition,  the  body  section  joints  on  the  first 
airplane  required  no  shims  and  the  aircraft’s  seat  tracks  lined  up  to  within  one- 
thousandths  of  an  inch  over  the  entire  length  of  the  fuselage. 

Moving  into  the  production  phase,  Boeing  used  M&S  technology  to  minimize 
production  costs  and  to  ensure  the  B-777  was  a  service-ready  product  upon  delivery  to 
the  airlines.  The  B-777  program  established  a  number  of  goals  to  address  this  issue.  One 
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of  the  first  program  goals  was  to  reduce  error  and  rework  in  the  manufacturing  process  by 
50  percent  by  using  M&S  technology  and  “design/build  teams.” 

Synthesis  of  Modeling  and  Simulation  Methods  with  DBTs 

In  order  to  achieve  the  goal  of  reducing  rework  and  error,  238  design/build  teams 
(DBTs)  made  extensive  use  of  CAD,  KBE,  and  concurrent  engineering  over  computer 
networks  so  that  design  changes  could  be  coordinated  easily.  DBTs  usually  consisted  of 
approximately  fifteen  members  with  representatives  from  design,  manufacturing,  quality 
assurance,  customer  service,  flight  operations,  and  flight  deck  crewmembers.  The  DBTs 
also  included  over  300  airline  employees  who  came  to  Boeing  in  order  to  make  inputs  to 
the  program.  This  was  an  important  organizational  change  for  the  Boeing  Company.  All 
the  parties  that  could  alter  the  design  or  affect  the  schedule  were  now  able  to  make  inputs 
at  the  very  beginning  of  the  design  process  instead  of  waiting  for  a  mockup  or  prototype. 

The  typical  team  met  every  few  weeks  to  monitor  progress  and  discuss  problems. 
Boeing  quickly  discovered  that  dividing  people  into  teams  did  not  assure  the  success  of 
the  DBT  concept.  Teams  must  have  effective  leadership  in  order  to  guide  the  process 
toward  a  decision.  Many  teams  were  great  at  discussion  but  never  arrived  at  engineering 
solutions.  Since  the  first  design  entered  into  the  computer  system  often  was  the  first  one 
to  be  “chopped”  or  criticized  by  the  other  teams,  initial  data  input  was  sometimes  quite 
slow.  Some  DBTs  also  tried  to  “game  the  system  by  over-designing  their  product  in  order 
to  have  some  room  for  design  tradeoffs  later.  Fortunately,  concurrent  engineering  and 
open  access  to  data  made  the  over-design  strategy  transparent  to  everyone  involved. 
Consistent  and  open  analysis  using  M&S  technology  meant  every  DBT’s  cards  were  on 
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the  table.  Therefore,  honest  coordination  and  sound  engineering  judgment  could 
eventually  prevail. 

In  order  to  ensure  a  smoothly  flowing  design  process,  senior  Boeing  management 
asked  the  DBTs  to  get  inputs  from  the  customers  into  the  design  process  and  then  enter 
these  inputs  as  critical  design  elements.  This  action  got  the  DBTs  into  direct  contact  with 
airline  personnel.  DBTs  were  sent  to  talk  directly  with  airline  mechanics  and  service 
personnel  in  order  to  get  their  point  of  view  on  the  designs.  One  specific  goal  was  to  find 
out  directly  from  the  gate  mechanics  what  information  they  needed  to  efficiently  service 
the  airplane. 

Two  examples  of  the  knowledge  gained  from  the  field  illustrate  the  value  of  human 
experience  and  some  limitations  of  M&S  alone.  In  one  of  the  electronics  bays  there  is 
light  centered  directly  in  the  ceiling.  One  of  the  mechanics  pointed  out  that  whenever  he 
was  the  bay  and  leaned  over  to  work,  he  inevitably  blocked  the  light  and  had  to  use  a 
flashlight.  The  simple  solution  was  to  install  lights  at  the  corners  of  the  enclosure. 
However,  as  the  result  of  this  seemingly  insignificant  observation,  Boeing  realized  that 
M&S  could  not  adequately  predict  human  factors  issues  and  design  criteria.  Lighting  and 
illumination  could  be  added  to  the  aircraft  model,  but  only  after  human  factors  studies 
discovered  the  illumination  issue. 

Another  example  came  when  the  team  opened  a  lower  hold  to  check  on  a  part  and 
discovered  hundreds  of  small  salt  packets  that  had  fallen  off  food  trays  and  found  their 
way  down  below  the  passenger  compartment.  The  combination  of  salt  and  moisture 
condensing  on  the  inner  skin  of  the  aircraft  produced  a  corrosive  salt-water  bath 
splashing  about  on  the  inside  of  the  aircraft.  This  may  seem  like  a  trivial  matter,  but  it  is 
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one  of  the  issues  Boeing  is  struggling  with.  How  to  get  M&S  to  anticipate  the  problems 
that  can  and  do  happen  in  the  real  world?  M&S  cannot  provide  field  service  data  or 
practical  servicing  tips.  Human  experience  with  the  system  is  the  only  way  to  discover 
these  problems. 

The  DBTs  had  to  gather  and  study  this  information  using  the  old  design 
methodologies  and  then  synthesize  what  they  learned  into  knowledge  the  designers  could 
actually  use.  In  the  past,  middle  managers,  sometimes  derisively  known  as  “gray-beards”, 
would  monitor  the  design  process  and  check  for  problems.  Although  with  M&S 
technology,  the  aircraft  designers  were  much  more  capable  of  finding  problems  and 
coordinating  requirements  without  as  much  management  oversight.  However,  M&S  did 
not  prevent  the  designers  from  making  errors  of  omission.  Design  requirements  make 
more  sense  to  the  individual  workers  when  the  information  was  part  of  a  model  instead  of 
an  abstract  and  seemingly  arbitrary  requirements  list.  Nevertheless,  the  combination  of 
DBTs  along  with  M&S  technologies  did  lead  to  a  mature  and  stable  design  much  earlier 
than  the  B-757  program  before  it.  Consequently,  the  B-777  test  and  certification 
processes  also  went  much  more  smoothly.  Flight-testing  went  according  to  schedule  with 
no  major  delays  for  either  redesign  or  engineering  analysis. 

Engineering  Results 

One  of  Boeing’s  key  goals  for  the  B-777  was  "Day  One  Service  Readiness",  by 
conducting  extensive  testing  and  integration  throughout  the  program.  Boeing  used  M&S 
technology  as  an  integral  part  of  the  ground-te sting  program  to  help  work  out  the  flaws 
before  production.  The  500,000  square  foot,  $110  million  Integrated  Aircraft  Systems 
Laboratory  (IASL)  is  the  newest  addition  to  Boeing’s  test  facilities.  It  contains  over  $250 
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million  of  test  equipment.  According  to  Boeing  engineer  Chris  Adams,  the  IASL  is  an 
example  of  the  philosophy  shift  from  "repairing"  toward  "planning"  in  the  B-777  effort. 

Aircraft  systems  that  in  the  past  had  not  been  tested  together  until  the  first  flight  were 
integrated  and  tested  in  the  IASL.  "Aircraft  Zero"  or  the  "B-777  without  the  skin",  as  the 
integrated  systems  were  called,  had  been  "flown"  or  simulated  for  many  hours  before  the 
first  aircraft  flight  over  Seattle.  This  testing  was  useful  in  moving  developmental  work 
forward  and  allowed  early  ETOPS  certification  by  performing  statistical  analysis  of  mean 
time  between  failures  for  the  digital  avionics  suite. 

The  FA  A  required  a  full-scale  failure  test  of  the  wings  on  the  second  production 
airplane  and  the  results  were  quite  impressive.  The  test  results  validated  the  CATIA 
component  level  analysis  almost  exactly.  Over  2,000  individual  strain  gauges  were  used 
to  instrument  the  plane  with  500  miles  of  cables  relaying  the  data  to  the  CATIA  system  to 
validate  the  structural  analysis  model.  Each  wing  was  subjected  to  500,000  pounds  of 
force  and  the  wing  tips  deflected  24  feet  before  failing  at  154  percent  of  maximum  flight 
loads.  This  applied  force  represented  a  factor  of  safety  of  1.5  times  the  loads  experienced 
during  a  2.5-g  recovery  from  a  dive.  During  the  test,  the  right  wing  failed  0.020  seconds 
before  the  left  wing.  This  is  just  above  the  152  percent  target  design  criteria  and  almost 
exactly  as  predicted  by  the  CATIA  engineering  model  of  the  completed  wing  assembly. 

The  B-777  also  had  the  most  extensive  flight-test  program  of  any  previous  Boeing 
airplane.  Nine  airplanes  were  used  with  engines  from  three  manufacturers.  Before  the  test 
program  was  completed,  all  nine  airplanes  flew  nearly  7,000  hours  for  more  than  2,900 
total  flights.  The  flight-testing  program  lasted  about  two  years  with  customer 
involvement  of  flight  and  ground  crews  in  the  latter  stages  of  the  test  program.  All  of  the 
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B-777s  involved  in  testing  carried  large  amounts  of  instrumentation  and  data  recording 
equipment.  The  data  collected  in-flight  was  used  to  validate  the  aircraft  model — as  a 
complex  system — developed  during  the  design  and  production  process. 

Handling  qualities  testing  during  the  first  two  months  of  the  flight-test  program 
proved  the  value  of  the  simulation  and  analysis  of  the  digital  flight  control  system.  Very 
few  changes  were  made  in  the  flight  control  system  software  as  the  result  of  in-flight 
handling  qualities  evaluations.  However,  the  M&S  effort  was  not  perfect  and  small 
adjustments  to  pitch  control  gains,  turn  coordination  scheduling,  and  the  flaperon 
commands  were  made  as  the  result  of  flight-test. 

M&S  also  did  not  predict  pilot  induced  oscillation  (PIO)  problems  that  were 
uncovered  early  in  the  flight-test  program.  The  most  significant  event  occurred  when  a 
new  test  pilot  was  checking  out  in  the  B-777.  During  the  de-rotation  after  landing,  a  pitch 
PIO  developed  which  went  for  at  least  three  cycles  and  became  a  flight  hazard  before 
intervention  by  the  other  pilot.  This  PIO  was  the  result  of  a  rapid  application  of  elevator 
followed  by  an  equally  rapid  check  that  led  to  the  elevator  actuator  becoming  saturated 
by  control  inputs.  Some  unexpected  oscillations  were  also  encountered  during  autopilot 
mode  transitions  and  mistrim  takeoff  conditions  with  large  and  abrupt  elevator 
commands.  However,  the  oscillation  tendencies  were  easily  controllable  by  the  pilot. 

Another  oscillatory  mode  was  discovered  during  flight-test  that  was  not  predicted  by 
the  M&S  of  the  aircraft  before  flight-testing.  This  oscillation  was  classified  as  Airplane- 
Pilot-Coupling  (APC).  This  was  a  three-Hertz  (Hz)  body-bending  mode  triggered  by 
rapid  pitch  commands.  This  mode  was  most  often  encountered  as  pilots  increased  the 
frequency  of  their  control  inputs  as  the  aircraft  entered  the  landing  flare.  This  oscillatory 
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mode  was  not  a  hazard  to  flight  safety,  and  was  quickly  fixed  with  software  changes, 
which  dampened  the  three  Hz  feedback  to  the  control  system.  In  the  end,  the  flight-test 
team  did  recommend  that  some  additional  work  be  done  in  modeling  man-in-the-loop 
oscillatory  motions.  The  flight-test  team  also  recommended  that  the  limitations  of 
modeling  complex  systems  be  incorporated  into  the  design  requirements  for  the  next 
generation  of  Boeing  aircraft. 

Fortunately,  the  Boeing  test  team  had  extensive  experience  with  M&S,  aircraft  flight 
control  models,  and  variable  stability  aircraft.  Because  M&S  was  an  integral  part  of  the 
engineering  and  design  effort  of  the  B-777  program  from  the  very  start,  the  flight-test 
team  better  understood  both  the  capabilities  and  limitations  of  M&S.  The  flight-test  team 
was  then  able  to  make  sound  recommendations  to  the  design  engineers.  Both  the  pilots 
and  the  project  engineers  realized  that  M&S  could  not  anticipate  all  the  engineering 
deficiencies  of  a  complex  system  such  as  a  modern  jet  aircraft.  The  combination  of  flight¬ 
testing  and  modeling  is  required — especially  as  systems  become  more  complex. 
Modeling  provides  prediction  and  analysis;  flight-testing  provides  real-world  engineering 
data  and  model  validation. 

Evaluation  M&S  for  Flight-Testing  Complex  Systems 

The  aerospace  industry  is  increasingly  becoming  an  information  processing  industry. 
Computers  have  moved  well  beyond  basic  control  functions.  Instead,  they  can  create  a 
“virtual”  prototype,  which  is  then  assembled  and  tested  in  cyberspace.  Managing  data  to 
make  design,  production,  and  testing  less  expensive  is  a  prerequisite  to  sound  engineering 
and  design  practices.  The  “airplane  in  a  computer”  concept  pioneered  by  the  B-777 
program  has  become  the  baseline  for  future  aircraft  design  programs  at  Boeing. 
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However,  the  costs  of  building  an  “an  airplane  in  a  computer”  are  still  quite 
substantial.  A  CATIA  workstation  with  the  appropriate  software  costs  approximately 
$10,000.6_  Therefore,  2,200  workstations  cost  the  company  a  little  more  than  $22  million. 
Although,  the  specific  costs  of  the  mainframe  computing  cluster  and  data  storage 
facilities  were  not  available  for  this  research,  they  added  significantly  to  the  overall  costs 
of  the  program. 

The  costs  of  M&S  hardware  and  software  dedicated  to  the  B-777  program  were  well 
above  the  $22.5  million  dollars  required  to  build  a  non-flying  prototype.  If  skipping  the 
prototype  phase  was  the  only  benefit  of  investing  in  M&S  technology,  then  M&S  would 
not  be  cost  effective.  However,  the  non-flying  prototype  is  a  sunk  cost.  There  are  no 
follow-on  benefits  and  the  prototype  eventually  must  be  scrapped.  On  the  other  hand, 
M&S  hardware  and  software  has  a  useful  life  of  three  to  five  years  before  becoming 
obsolete  and  can  be  used  for  other  programs.  The  IASL  was  a  $1 10  million  investment  in 
M&S,  but  it  will  serve  the  needs  of  all  Boeing’s  aircraft  programs  for  many  years  to 
come.  M&S  also  permits  aircraft  design  data  to  be  archived  easily  and  inexpensively  so 
that  costly  lessons  learned  one  design  and  test  program  can  be  used  on  the  next 
generation  of  aircraft. 

Therefore,  the  costs  of  M&S  for  a  complex  system  cannot  be  easily  justified  for  a 
single  program.  The  commitment  to  model  a  complex  system  brings  about  significant 
changes  to  the  way  an  entire  organization  does  business.  Boeing  is  in  the  business  of 
designing  and  building  airplanes  and  needs  to  cut  costs  while  simultaneously  staying 
ahead  of  the  competition  technologically.  M&S  can  only  be  cost  effective  for  a  complex 

Note 

62  These  figures  are  the  best  available  estimates  as  of  1  May  1998  from  the  Boeing 
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system  such  as  the  B-777,  when  the  company  is  willing  to  commit  to  M&S  for  the  long 
term.  The  company  must  also  be  willing  to  accept  the  organizational  changes  required  to 
integrate  M&S  into  the  entire  design,  production,  and  testing  process. 

In  order  to  ensure  quality  during  the  design  and  testing  process,  M&S  must  provide 
valid  engineering  data.  In  the  case  of  the  B-777,  the  engineering  validity  of  the  models 
was  excellent.  All  testing  of  the  aircraft  at  the  component  level  showed  outstanding  form, 
fit,  and  function.  The  accuracy  achieved  with  M&S  was  simply  unattainable  with 
standard  engineering  and  design  methods.  In  fact,  the  first  production  B-777  was  also  the 
first  aircraft  ever  assembled  by  Boeing  that  did  not  require  shimming  when  the  fuselage 
was  constructed.  Likewise,  the  wings  alignment  was  within  the  accuracy  of  the 
instrumentation.  All  ground  and  flight-test  data  at  the  component  level  matched  M&S 
predictions.  Therefore,  the  engineering  validity  of  the  M&S  data  was  excellent — even  for 
a  complex  system  such  as  the  B-777. 

Nevertheless,  Boeing  did  push  the  technological  boundaries  with  the  B-777  program. 
There  were  some  problems  at  the  systems  level  with  the  comprehensiveness  of  the  M&S 
effort.  For  example,  the  PIO  problem  encountered  during  flight-test  was  not  predicted  by 
any  systems  level  model  of  the  aircraft.  In  addition,  the  APC  difficulties  encountered 
during  landing  were  completely  unforeseen  by  M&S.  Fortunately,  Boeing’s  flight-test 
team  had  extensive  experience  with  both  digital  flight  controls  and  the  hazards  associated 
with  flight-testing  these  systems.  These  problems  were  easily  fixed,  but  only  because 
sound  engineering  judgments  were  made  by  experienced  people.  M&S  augmented  but 
did  not  replace  the  human  element  of  designing  and  testing  aircraft. 

Notes 

Commercial  Aircraft  Corp.,  courtesy  of  the  Boeing  767division. 
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The  aerospace  industry’s  dependency  on  M&S  technology  is  mushrooming. 
Complex  systems  models  are  not  yet  comprehensive  tools  for  designing  and  testing 
airplanes.  The  problem  of  building  and  maintaining  comprehensive  models  is  being  made 
even  more  difficult  by  rapidly  changing  information  technologies.  Boeing  estimates  that 
only  one-third  the  number  of  software  applications  in  use  today  will  be  here  in  another 
fifteen  years.63  Cristoph  W.  Klomp,  who  leads  Boeing’s  737  software  office,  says:  “This 
company  would  collapse  if  there  were  no  IT  (information  technology)  services,  but 
exploration  of  these  services  is  still  very,  very  shallow.”64  Boeing  has  found  that  despite 
the  remarkable  improvements  in  M&S  technology,  M&S  still  has  significant  limitations 
when  modeling  complex  systems.  Computer  problems  still  vex  projects  temporarily  and 
delay  programs,  and  even  “expert”  systems  cannot  provide  sound  engineering  judgment. 

Frank  McCormick,  the  lead  engineer  for  the  B-777  autopilot  system,  said:65 

There  are  utilities  and  tools  that  have  been  around  for  a  long  time  that  help 
with  certain  aspects  of  the  job... with  the  bookkeeping  aspects... What 
everyone  wants  and  no  one  has  found  is  a  magic  utility  that  will  help  you 
develop  a  safe,  complete,  unambiguous,  and  non-conflicting  requirements. 

That  is  still  requires  human  experience  and  sound  engineering  judgment.  Computers 
only  aid  the  designers  and  store  the  data.  Flight-test  will  still  be  required  and  the 
engineering  decision  process  is  still  very  much  a  human  activity. 


Note 

63  Michael  Mecham,  "Boeing  Translates  737  to  Digital  Age,"  Aviation  Week  and  Space 
Technology  Vol.  147  no.  14  (Oct.  6,  1997):  (48-49). 

64 


65 


ibid. 

Mecham,  "Aerospace  Chases  the  Software  Boom”  48. 
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Chapter  5 


The  Future  of  Modeling  and  Simulationfor  Flight  Test 


The  day  is  rapidly  approaching  when  a  significant  portion  of  aircraft  design  and 
engineering,  especially  the  test  and  evaluation  phase,  will  be  replicated  in  an  artificial 
environment.  Advances  in  computer  technology  are  making  this  possible,  and  declining 
resources  are  making  it  mandatory.  M&S  is  still  expensive,  but  not  prohibitively  so. 
Moreover,  the  real  costs  of  M&S  are  declining  while  capabilities  are  rapidly  increasing. 

Currently  the  capabilities  of  M&S  are  doubling  approximately  every  eighteen 
months.  This  is  the  case  for  three  reasons.  First,  the  processing  speeds  of  computers  are 
doubling  about  every  twenty-four  months  and  are  expected  to  continue  to  do  so  in  the 
future.  Second,  the  modeling  techniques  and  computational  methods  are  becoming  more 
sophisticated.  We  can  now  model  complex  physical  phenomenon  such  as  turbulent 
flowfields  that  were  not  well  understood  only  a  few  years  ago.  Third,  the  modeling 
efforts  used  in  one  program  usually  are  easily  importable  into  another  M&S  project. 

The  three  factors  of  increased  computational  power,  more  sophisticated 
computational  methods,  and  cross-platform  applicability  of  computational  techniques  are 
causing  these  extraordinarily  rapid  advances  in  M&S  technologies.  The  speed  of 
technological  advance  is  changing  the  way  systems  are  developed  and  fielded.  M&S  is 
now  a  critical  aspect  of  most  major  weapons  systems  development.  The  flight-test 
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community  must  be  proactive  in  utilizing  this  new  tool.  The  ability  to  conduct  effective 
M&S  must  be  developed  as  an  integral  part  of  future  flight-test  programs. 

Evaluation  of  Cost,  Engineering  Validity,  and  Comprehensiveness 

All  three  programs  discussed  in  this  paper  have  made  significant  progress  towards 
meeting  the  challenges  set  out  by  senior  Air  Force  leadership  in  A  New  Vector  for  Air 
Force  Modeling  and  Simulation.  Although  M&S  is  not  free,  it  is  almost  always  more 
cost-effective  than  testing  alone  when  applied  at  the  physical  component  or  the  closed- 
loop  system  level.  It  can  be  significantly  more  cost-effective  than  traditional  engineering 
techniques  when  applied  to  a  complex  system  such  as  the  B-777,  if  the  organization  is 
willing  to  commit  to  technological  and  organizational  changes. 

The  engineering  validity  of  M&S  was  excellent  in  all  three  cases  studied  during  this 
research.  The  accuracy  of  the  models  was  equal  to  or  better  than  traditional  engineering 
methods.  The  variance  between  flight-test  data  points  and  the  M&S  predictions  were 
barely  measurable  by  the  flight-test  instrumentation.  For  example,  the  B-1B  flare 
trajectory  predictions  were  within  one  foot  of  the  flight-test  data  points.  This  was  the 
most  accurate  result  possible,  because  the  intense  brightness  of  the  burning  flare  saturated 
the  film  in  a  one  foot  circle  around  the  flare.  Therefore,  the  instrumentation  accuracy  was 
plus  or  minus  one  foot,  which  was  the  same  accuracy  as  the  trajectory  model. 

In  the  case  of  the  VISTA,  there  were  very  few  measurable  differences  between  the 
M&S  data  points  and  the  flight-tested  data  on  the  F-22.  However,  the  VISTA  did  not 
have  as  large  a  flight  envelope  as  the  F-22  and  the  sampled  data  from  the  VISTA  had 
slightly  different  statistical  characteristics  than  the  F-22  data  taken  over  the  complete 
flight  envelope.  Likewise  with  the  B-777,  the  individually  modeled  data  points  were 
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extraordinarily  accurate.  Component  parts  were  so  accurately  engineered  that  no  aircraft 
prototype  was  required  to  check,  form,  fit,  and  function  before  production.  Therefore 
M&S  was  more  accurate  than  traditional  engineering  methods  for  the  B-777  program  at 
the  physical  component  level,  but  the  computer  model  had  a  limited  range  of  accurate 
predictions.  The  limitations  of  M&S  become  more  significant  as  the  model  increased  in 
scope  and  complexity. 

A  modeling  effort  often  increases  in  scope  and  complexity  by  building  upon 
previous  M&S  work.  Engineering  teams  often  take  the  computational  methods  and  data 
used  on  a  previous  project  and  update  them  for  the  task  at  hand.  This  was  certainly  the 
case  in  the  B-1B  flare  strike  project.  The  team  began  by  taking  B-1B  models  from  earlier 
programs  and  modifying  them  to  suit  project  requirements.  Aeronautical  data  from  earlier 
projects  were  applied  to  the  flare  trajectory  model  and  spin-off  technologies  from  the 
F-22  program  were  also  used.  Despite  the  myriad  of  sources  used  to  construct  the  B-1B 
flare  trajectory  model,  it  completely  and  accurately  predicted  the  engineering  outcomes 
of  flight-test.  Because  the  scope  of  the  B-1B  modeling  effort  was  limited  to  the  physical 
component  level,  the  model  was  provided  comprehensive  solutions  to  the  problem  at 
hand. 

In  the  case  of  the  VISTA,  the  M&S  effort  provided  comprehensive  solutions  within 
the  limited  flight  envelope  of  the  VISTA  simulator.  There  were  no  unexpected  results 
from  the  VISTA  flight  tests.  The  F-22  response  matched  the  VISTA’s  almost  exactly 
within  a  limited  but  known  range.  Nevertheless,  it  was  possible  to  draw  invalid 
engineering  solutions  from  the  VISTA  by  trying  to  extrapolate  results  by  statistical 
analysis  beyond  the  valid  range  of  the  model.  The  actual  F-22  parameter  data  had  slightly 
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different  statistical  distributions  than  the  VISTA  data  because  the  VISTA  was  only 
capable  of  sampling  flight  parameters  in  a  more  restrictive  flight  envelope. 

However,  the  B-777  system  was  so  complex  that  the  model  was  not  comprehensive 
enough  to  predict  all  the  significant  engineering  results  from  flight-test.  Minor  changes  to 
flight  control  software  were  required  after  flight-testing  and  some  unexpected  modes  of 
motion  were  discovered  in  test  that  were  not  predicted  from  M&S.  These  unexpected 
modes  of  motion  did  not  cause  a  hazard  during  flight-test,  because  the  test  crews  were 
aware  of  the  limitations  of  M&S  for  a  sophisticated  project  such  as  the  B-777.  The  crew 
knew  the  theory  of  digital  control  systems  and  understood  the  potential  hazards. 
However,  the  unexpected  results  might  have  been  a  severe  hazard  for  a  less  experienced 
crew.  The  risk  of  modeling  without  flight-testing  is  too  great  to  rely  completely  on  M&S 
for  complex  systems.  Figure  15  below  summarizes  the  evaluation  of  costs,  engineering 
validity,  and  comprehensiveness  for  each  of  the  cases  in  this  research. 


B-1B  Flare  VISTA  Boeing  777 

Physical  Comp.  Closed-Loop  Model  Complex  System 
Model  Model 


Costs 

$950,000  with 

CFD  modeling 

$3.5  million  without 
modeling 

VISTA  modeling  at 
$8, 000/hr. 

F-22  flight-test  at 
$20, 000/hr. 

Approx.  $250 

million  long-term 

M&S  investment 

$22.5 

million/prototype 
without  modeling 

Within  flight-test 

Within  flight-test 

Errors  smaller  than 

Engineering 

instrumentation 

instrumentation 

standard  engineering 

Validity 

accuracy 

accuracy 

methods 

Completely 

Completely 

Not  comprehensive 

Comprehensive 

comprehensive 

comprehensive 
within  a  limited  but 
known  range 

because  some  flight- 
test  results  not 

predicted  by  M&S 

Figure  15:  Evaluation  of  Case  Results 
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Lessons  Learned  for  Future  Flight-Testing 


Four  very  important  lessons  can  be  learned  for  the  future  of  flight-test  from  these 
three  case  studies.  First,  changes  in  tactical  doctrines  for  a  weapon  system  can  generate 
new  flight  test  requirements  for  that  system.  This  lesson  is  particularly  important  as 
weapon  systems  see  longer  life  spans  and  their  mission  requirements  sometimes  change 
radically.  Both  the  flight  test  community  and  the  operational  users  must  maintain  a 
constant  dialog  with  each  other  to  ensure  that  such  doctrine  changes  and  corresponding 
test  candidates  are  identified  as  early  as  possible. 

Second,  the  concept  and  application  of  integrated  test  and  evaluation  provides  rich 
dividends.  When  the  flight  testers,  ground  testers,  modelers,  simulators,  program 
managers,  and  vendors  proactively  pool  their  resources  throughout  the  design, 
production,  and  test  phases,  the  benefits  in  cost  and  schedule  reductions,  as  well  as 
performance  increases,  are  superb.66 

Third,  the  recent  leaps  in  technology  within  the  M&S  community,  and  the 
subsequent  maturation  of  the  capabilities,  have  provided  a  powerful  and  vital  tool  for  use 
in  many  future  flight  test  programs.  The  flight  test  community  should  make  a  point  to 
seek  out  these  capabilities  continually  in  the  design  of  every  flight  test  program.  M&S 
will  never  replace  the  need  for  flight  test,  especially  since  models  can  never  be  fully 
validated  without  flight  test  data,  but  M&S  can  certainly  assist  in  smarter,  more  informed 
test  planning  and  conduct.67 


Note 

66  Dr.  David  C.  Brown,  “A  Tester  Prepares  for  Synthetic  Acquisition,”  International  Test  and  Evaluation 
Association  Journal,  June/July  1996,14-17. 

67  Dr.  Patricia  Sanders,  “Modeling  and  Simulation:  Challenges  for  the  Test  and  Evaluation  Community” 
presentation  to  an  International  Test  and  Evaluation  Association  Workshop,  Las  Cruses,  New  Mexico,  13 
December  1995. 
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Finally,  the  design  of  flight  test  instrumentation  should  include  elements  required  for 
model  validation,  even  if  those  elements  are  not  immediately  required  for  the  flight  test 
itself.  The  Undersecretary  of  Defense,  Dr.  Paul  G.  Kiminski  has  endorsed  this  measure, 
in  a  speech  to  the  International  Test  and  Evaluation  Association.68  This  lesson  could 
easily  be  extrapolated  to  actually  include  objectives  in  the  flight  test  plan  to  provide 
model  validation  data.  With  a  constant  feedback  loop  established,  the  test  and  evaluation 
community  would  directly  invest  in  itself  with  A  New  Vector  for  its  future. 


Note 

6S  Honorable  Paul  G.  Kiminski,  Undersecretary  of  Defense  for  Acquisition  and  Technology,  Address  to  the 
International  Test  and  Evaluation  Symposium,  Huntsville,  AL,  3  October  1995. 
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Appendix  A 


Modeled  Safe  Deployment  Envelope  of  MJU-23B  Flares 
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Models  of  Flare  Ejection  Velocity  on  Flare  Deployment  Envelope 
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Appendix  B 


VISTA  Technical  Specifications 


Handling  Qualities  Simulation  Capabilities 


Short  Period  Natural  Freq. 

0  to  15  rad/sec 

Short  Period  Damping 

-0.1  to  1.1 

Nz/alpha 

1  to  lOOg/rad 

Stick  Force/g 

1  to  2001b  s/g 

Phugoid  Natural  Freq. 

0  to  .5  rad/sec 

Phugoid  Damping 

-0.3  to  1.0 

Dutch  Roll  Natural  Freq. 

2  to  8  rad/sec 

Dutch  Roll  Damping 

-0.1  to  1.0 

Roll/Spiral  Mode  Natural  Freq. 

0  to  5  rad/sec 

Roll/Spiral  Mode  Damping 

-0.1  to  1.0 

Roll/Sideslip 

0  to  10 

Variable  Time  Delay 

0.01  to  0.5  sec 

Lead/Lag 

1  to  63  rad/sec 

Aircraft  Operating  Envelope 

Sideslip 

Side  Acceleration 
Landing  Speed 
Direct  Lift 

Pitch  Pointing 

Max.  Normal  Acceleration  (g’s) 


+/-10  degrees 
+/-0.5g 
130-160  Kts 
+/-lg  in  0.2  sec 

+/-7  deg 

Delta  Alpha 

-2.4/+7.33 
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Appendix  C 


Boeing  111  Technical  Specifications: 


Length 

209'  1"  (64  meters) 

Wing  span 

199'  11"  (61  meters) 

Height 

Unknown 

Max  Take  Off  Weight 

535,000  lb.  (242,672  kg) 

Speed 

Mach  0.84 

Range 

4,353-5,332  miles  (3,780-4,630  nautical  miles) 

Armament 

None 

Crew 

Two  Pilots  plus  cabin  crew 

Unit  cost 

$128.0  to  170.0  million 

Inventory 

N/A 

Constructor 

Boeing 

81 


Glossary 


ACC 

Air  Combat  Command 

AEDC 

Arnold  Engineering  Development  Center 

AFFTC 

Air  Force  Flight  Test  Center 

AFMC 

Air  Force  Material  Command 

AGF 

Above  Ground  Fevel 

AFC 

Air  Fogistics  Center 

ATIMS 

Airborne  Turret  Infrared  Measurement  System 

ATIS 

Airborne  Test  Instrumentation  System 

CAD 

Computer  Aided  Design 

CATIA 

Computer-Aided,  Three-Dimensional  Interactive 
Application 

CFD 

Computational  Fluid  Dynamics 

CTF 

Combined  Test  Force 

DBT 

Design  Build  Team 

DOF 

Degrees  of  Freedom 

DSO 

Defensive  Systems  Operator 

DTUC 

Data  Transfer  Unit  Cartridge 

ETOPS 

Extended  Range  Twin-Engine  Operations 

fps 

feet  per  second 

GPS 

Global  Positioning  System 

HUD 

Heads  Up  Display 

IAS 

Indicated  Airspeed 

IASF 

Integrated  Aircraft  Systems  Faboratory 

KBE 

Knowledge  Based  Engineering 

FRU 

Fine  Replaceable  Unit 

M 

Mach  Number 

MDA 

Milestone  Decision  Authority 

MSF 

Mean  Sea  Fevel 
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NAWCWPNS  Naval  Air  Warfare  Center,  Weapons  Division 

NFOV  Narrow  Field  of  View 


OSO  Offensive  Systems  Operator 

PIO  Pilot  Induced  Oscillation 


SPO  Systems  Program  Office 

SSO  Systems  Sustainment  Office 

TAS  True  Airspeed 

VISTA  Variable  Stability  In-flight  Simulator  Test  Aircraft 

VSS  Variable  Stability  System 

WFOV  Wide  Field  of  View 


Algorithm.  A  step-by-step  problem-solving  procedure,  especially  an  established, 
recursive  computational  procedure  for  solving  a  problem  in  a  finite  number  of  steps. 

Alpha.  The  variable  commonly  assigned  in  aerodynamic  equations  to  angle  of  attack. 

Angle  of  attack.  The  angle  between  the  relative  wind  and  the  chord  line  of  an  airfoil. 

Beta.  The  variable  commonly  assigned  in  aerodynamic  equations  to  sideslip. 

Benchmark.  The  activity  of  comparing  the  results  of  a  model  or  simulation  with  an 
accepted  representation  of  the  process  being  modeled. 

Benchmarking.  The  comparison  between  a  model’s  output  and  the  outputs  of  other 
models  or  simulations,  all  of  which  represent  the  same  input  and  environmental 
conditions. 

Black  Box  Model.  A  model  whose  inputs,  outputs,  and  functional  performance  are 
known,  but  whose  internal  implementation  is  unknown  or  irrelevant.  For  example,  a 
model  of  a  computerized  change -return  mechanism  in  a  vending  machine,  in  the  form 
of  a  table  that  indicates  the  amount  of  change  to  be  returned  for  each  amount 
deposited. 

Boundary  Condition.  The  values  assumed  by  the  variables  in  a  system,  model,  or 
simulation  when  one  or  more  of  them  is  at  a  limiting  value  at  the  edge  of  the  domain 
of  interest. 

Built-in- Simulation.  A  special-purpose  simulation  provided  as  a  component  of  a 
simulation  language;  for  example,  a  simulation  of  a  bank  that  can  be  made  specific  by 
stating  the  number  of  tellers,  number  of  customers,  and  other  parameters 

Built-in-Simulator.  A  simulator  that  is  built-in  to  the  system  being  modeled;  for 
example,  an  operator  training  simulator  built  into  the  control  panel  of  a  power  plant 
such  that  the  system  can  operate  in  simulator  mode  or  in  normal  operating  mode. 

Chimera  overset  mesh  methodology.  Computational  techniques  using  simplified 
models  of  parts  of  a  complex  shape  connected  by  a  grid  or  coordinate  system. 
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Degree-of-freedom.  The  ability  of  a  free  body  in  space  to  move  by  either  translation  or 
rotation  along  an  axis.  Note:  six  degrees  of  freedom  implies  the  body  is  free  to  rotate 
and  or  translate  unconstrained. 

Empennage.  The  tail  section  of  an  airplane  to  include  both  the  horizontal  and  vertical 
stabilizers. 

Empirical.  Relying  on  or  derived  from  observation  or  experiment- 

Euler  equations.  First  order  differential  equation  relating  the  change  in  momentum  of  a 
gas  to  the  force. 

Gaussian  Distribution.  A  theoretical  frequency  distribution  for  a  set  of  variable  data, 
usually  represented  by  a  bell  shaped  curve  symmetrical  about  the  mean. 

In  viscid.  The  property  of  a  fluid  having  negligible  or  no  resistance  to  flow 

Monte  Carlo  methods 

Navier-Stokes  equations.  A  series  of  differential  equations  relating  the  change  in 
momentum  of  a  gas  to  the  forces  applied  accounting  for  resistance  to  flow  (viscosity) 
and  compressibility  of  the  gas. 

Sideslip.  The  angle  between  the  longitudinal  axis  of  an  aircraft  in  flight  and  the  direction 
of  the  airflow. 

Spectrometer.  An  instrument  used  to  determine  the  intensity  of  various  wavelengths  in  a 
spectrum  of.  light. 

Stochastic.  Involving  or  containing  a  random  variable  or  variables- 

Taylor  series  expansion.  An  approximation  of  the  solution  to  a  mathematical  function 
which  uses  an  infinite  series  of  derivatives  of  that  function. 

Transonic.  Of  or  relating  to  aerodynamic  flow  or  flight  conditions  at  speeds  close  to  the 
speed  of  sound. 
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